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PREFACE 

The  objective  of  the  seminar  was  to  provide  a  forum  for  senior  fleet  ISO's  to  review  the 
results  to  date  on  project  7754  of  the  Naval  Training  Equipment  Center.  Project  7754  Is  a  6.2 
R&D  effort  seeking  to  define  requirements  for  a  universal  LSO  waving  training  system.  The  tech¬ 
nologies  to  be  reviewed  were  development  of  Behavioral  Objectives.  Having  Decision  Making  Models. 
Performance  Assessment  Techniques,  Basic  System  Design  concepts,  and  Laboratory/Demonstration 
Training  Systems. 

The  final  efforts  under  this  project  will  be  development  of  models  of  critical  pilot  behavior 
and  of  Instructor  behavior.  Pilot  models  will  be  used  to  generate  specific  example  situations. 
These  critical  situations  are  expected  to  teach  the  LSO  to  recognize  key  factors  leading  to  waving 
decisions.  Instructor  models  will  provide  management  of  the  various  subsystems  by  managing  the 
Information  within  the  training  system  and  presenting  It  to  the  human  Instructor  In  such  a  way  as 
to  provide  graceful  man/machine  Interaction.  Of  particular  Interest  will  be  cost  effectiveness 
estimates  from  a  review  of  accident  reports  during  the  model  development  effort. 
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INSTRUCTIONAL  RESEARCH  AND  TRAINING  SYSTEMS 
FOR  LANDING  SIGNAL  OFFICERS 

DR.  HENRY  M.  HALFF 
Office  of  Naval  Research 


The  purpose  of  this  presentation  is  to 
inform  a  rather  diverse  audience  of  the  Office 
of  Naval  Research's  (ONR)  interest  In  this 
seminar  and  in  the  Naval  Training  Equipment 
Center's  (NTEC)  plans  for  Landing  Signal 
Officer  (LSO)  training.  The  simple  explanation 
for  the  seminar  is  that  it  struck  Robert  and 
myself  as  a  promising  way  of  exploring  the 
applied  aspects  of  ONR's  research  in  percep¬ 
tion  and  cognition.  In  the  rest  of  this  paper 
I  will  try  to  explain  why  we  saw  some  poten¬ 
tial  in  interfacing  our  two  programs  and  why  we 
chose  this  particular  mechanism.  I  will  begin 
with  a  few  words  on  ONR's  mission  and  relevant 
research,  a  topic  on  which  I  am  fairly  well 
informed.  I  will  then  say  a  good  deal  more 
about  the  application  of  ONR  research  to  NTEC's 
LSO  training  program,  but  you  should  know 
that  my  knowledge  of  this  program  is  small 
indeed  as  is  my  knowledge  of  the  LSO's  job 
itself.  In  fact,  the  sum  total  of  my  infor¬ 
mation  comes  from  Logicon's  design  study  on 
the  subject  (Hooks,  Butler,  Cullen  &  Petersen, 
1978). 

As  I  mentioned,  I  stand  on  firmer  ground 
when  speaking  about  ONR.  To  quote  from  our 
own  description  of  the  contract  research  and 
technology  program  (Office  of  Naval  Research, 
1978),  ONR  sponsors  "long-range  scientific 
research  believed  to  offer  potential  for  the 
advancement  and  improvement  of  naval  opera¬ 
tions."  Let  me  comment  on  a  couple  of  ambig¬ 
uities  in  this  statement.  First,  the  term 
long-range  is  interpreted  in  our  office  as 
being  in  the  neighborhood  of  five  to  fifteen 
years  away  from  application.  Hence,  one  of 
the  questions  which  I  should  answer  Is  how 
ONR  became  involved  in  the  LSO  training  pro¬ 
ject,  given  any  reasonable  expectations  about 
the  pace  of  NTEC's  development  of  the  work. 
Another  question  about  the  statement  is  "Who 
does  the  believing  in  this  research?"  Offic¬ 
ially  this  faith  is  distributed  throughout 
ONR's  official  chain  of  command,  but  in  fact 
the  major  share  of  the  responsibility  for  our 
research  rests  in  people  like  myself.  A  first 
look  at  ONR  reveals  something  very  much  like 
a  university,  with  departments  organized 
around  academic  disciplines  and  professionals 
who  are  given  considerable  latitude  in  decid¬ 
ing  what  research  should  be  done  within  these 
disciplines.  Here,  however,  the  resemblance 
to  a  university  ends  since  we  must  be  respon¬ 
sive  to  an  entity  called  the  future  Navy,  a 
fact  not  often  appreciated  by  either  the 
present  Navy  or  by  our  contractors.  We  try 
to  choose  research  areas  which  will  1)  address 
the  particular  problems  which  this  unseen 
organization  will  face,  e.g.,  fewer  and  less- 


intelligent  people  and  more  complex  equipment; 

2)  afford  greatest  leverage  with  our  current 
limited  resources;  and  3)  represent  high  qual¬ 
ity  basic  research.  To  expand  this  descrip¬ 
tion  by  one  level,  the  departments  in  our 
"university",  called  divisions  at  ONR,  include 
three  very  applied  research  areas,  which  I  need 
not  mention,  and  six  basic-research  areas: 
Physical  Sciences,  Mathematical  and  Informa¬ 
tion  Sciences,  Biological  Sciences,  Arctic 
and  Earth  Sciences,  Material  Sciences,  and 
Psychological  Sciences,  the  smallest  but  most 
important  division.  I  claim  this  importance 
because  the  Navy  now  spends  most  of  its  money 
not  on  planes,  ships,  and  weapons,  but  on 
people. 

Within  Psychological  Sciences,  we  support 
several  types  of  research  relevant  to  this 
seminar.  We  are  divided  into  three  branches: 
Organizational  Effectiveness,  Engineering 
Psychology,  and  my  branch.  Personnel  and  Train¬ 
ing.  The  Organizational  Effectiveness  Programs 
might  make  a  contribution  to  the  team  aspects 
of  LSO  training,  but  since  this  is  not  the 
primary  thrust  of  NTEC's  project,  Robert  did 
not  request  the  presence  of  their  representa¬ 
tives  here.  Our  Engineering  Psychology  Pro¬ 
grams  could  have  much  to  contribute  to  this 
work.  Perhaps  the  most  pressing  need  which 
they  could  address  is  for  some  guidance  on  the 
crucial  display  issues  for  LSO  training  devices, 
and  Robert  suggested  that  we  include  some  con¬ 
tractors  working  in  that  area.  But  the  bulk 
of  contractors  at  this  conference  are  repre¬ 
sentatives  of  the  Personnel  and  Training 
Research  Programs.  For  some  time  now,  much 
of  the  research  sponsored  by  these  programs 
has  been  focused  on  the  advanced  use  of  com¬ 
puters  in  testing,  education  and  training. 

Some  of  that  work,  on  computerized  adaptive 
testing,  is  not  particularly  relevant  here 
except  as  a  possible  source  of  mathematical 
models  for  proficiency  assessment  within  the 
system.  The  rest  of  our  research,  however, 
has  a  strong  cognitive  and  information- 
processing  orientation,  and  it  is  the  most 
appropriate  of  that  work  which  Robert  and  I 
chose  to  have  represented  here. 

Some  of  our  most  relevant  research  seems, 
oddly  enough,  to  be  in  the  area  of  individual 
differences  in  information-processing  ability. 

It  happens  to  be  important  in  this  context 
because  we  have  particularly  looked  at  indiv¬ 
idual  differences  in  heavily  time-driven  tasks. 
In  the  course  of  this  work  we  have  had  the 
opportunity  to  explore  many  issues  relevant  to 
LSO  training  including  automatlcity ,  attention- 
al  flexibility,  and  time-sharing  capacity. 
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Another  part  of  our  program,  called  cognitive 
processes.  Is  concerned  with  the  cognitive 
analysis  of  highly-advanced  Information¬ 
processing  skills.  Some  of  these  skills  con¬ 
sist  of  fairly  slow  problem-solving  and 
analysis  activities,  but  others  require  the 
accurate  perceptions  and  fast  reactions 
characteristic  of  the  LSO's  job. 

Perhaps  the  most  relevant  part  of  our 
research  for  this  seminar  is  that  on  Instruc¬ 
tional  Theory  and  Advanced  Training  Systems. 
This  work  is  the  most  recent  direction  of  our 
long-standing  interest  in  education  and  train¬ 
ing  applications  of  what  is  now  called  cogni¬ 
tive  science.  Our  earlier  work  in  the  field 
was  addressed  to  automated  tutors  for  educa¬ 
tional  environments,  but  over  the  past  couple 
of  years  we  have  shifted  the  research  to  deal 
with  procedural  knowledge  and  training  systems. 
As  with  Cognitive  Processes,  some  of  that  work 
is  directed  to  training  problem-solving  skills, 
but  much  is  directed  explicitly  to  the  in¬ 
structional  foundations  of  training  in 
devices  such  as  those  being  developed  here  at 
NTEC .  Maybe  part  of  the  answer  to  the  future- 
Navy  question  lies  here.  For,  as  the  Hooks 
et  al .  (1978)  and  Chatfield,  Marshall  and 
Gidcumb  (1979)  point  out,  these  devices  will 
take  on  more  and  more  of  the  training  burden 
in  the  Navy,  and  little  is  known  about  the 
proper  use  of  even  the  training  devices  now 
available;  current  flight  simulators,  for 
example,  are  often  used  for  little  more  than 
aircraft  surrogates. 

Hooks  et  al  .  also  point  out  many  of  the 
important  questions  which  we  hope  to  address 
in  our  own  research  programs.  What  is  the 
best  way  of  representing  proficiency  and 
advances  therein?  Is  it  possible  to  create 
computational  models  of  these  representations? 
Can  performance  data  from  simulators  be  used 
to  assess  proficiency  within  such  models? 

The  many  parameters  of  particular  exercises 
need  to  be  determined  at  each  point  in  train¬ 
ing  as  a  function  of  the  syllabus,  proficiency 
assessments,  and  instructor  input;  is  there  a 
theory  which  can  guide  this  process?  Finally, 
there  are  questions  concerning  the  use  of 
special  instructional  features  such  as  voice 
technology,  playback,  and  predictive  displays. 
All  Indications  are  that  in  today's  operation¬ 
al  training  devices  these  features  are  little 
used  and  less  understood. 

Borrowing  Lord  Hillary's  words,  I  might 
comment  that  this  ISO  training  program  offers 
itself  as  an  opportunity  to  consider  these 
issues  simple  "because  it  is  there."  But 
there  are,  in  addition,  several  features  of 
the  program  which  make  It  an  attractive 
scenario.  For  one  thing,  the  training  environ¬ 
ment  Is  limited  in  Important  ways.  The  task 
itself  is  limited  In  time  to  the  few  seconds 
required  to  get  the  aircraft  on  the  deck  or 


wave  her  off.  Likewise,  the  standard  response 
repertoire  is  limited  to  a  finite,  even  small, 
number  of  voice  commands  and  switch  closures. 
Also,  the  training  environment  itself  is  limit¬ 
ed  to  a  configuration  which  resembles  a  psycho¬ 
logical  laboratory  more  than  the  LSO  workspace 
onboard  a  carrier. 

This  point  is  related  to  a  second  import¬ 
ant  aspect  of  the  device,  namely,  experimental 
control .  I  have  often  been  chided  by  educa¬ 
tional  psychologists  who  claim  to  be  working  in 
the  real  world  (i.e.,  classrooms)  while  I 
puttered  away  in  my  fancy  computer-controlled 
laboratory,  which  allowed  me  to  present  an 
infinite  variety  of  precisely  timed  stimuli 
and  collect  responses  of  individuals  with  equal 
precision.  Thanks  to  Robert  and  those  working 
with  him,  I  have  the  last  laugh,  for  that 
laboratory  situation  has  now  become  the  real- 
world  instructional  environment.  The  require¬ 
ments  of  flexibility  to  meet  training  needs 
also  happen  to  provide  stimulus  control  for 
experimental  needs.  The  response  measurements 
required  for  a  responsive.  Interactive  training 
environment  are  precise  enough  also  to  provide 
hard  scientific  data.  The  potential  of  these 
devices  for  precise  instructional  research  is 
tremendous,  and  if  such  research  produces  less 
than  satisfactory  results,  we  cannot  complain 
of  a  lack  of  control  over  the  instructional 
environment. 

But  perhaps  the  most  important  aspect  of 
this  LSO  training  project  and  related  efforts 
is  the  relevance  of  certain  psychological 
issues  which  are  the  focus  of  much  investiga¬ 
tion  in  ONR  research  programs.  These  issues 
start  slightly  behind  the  eyeball  for,  as  I 
mentioned,  one  of  the  central  questions  in  this 
work  is  the  best  way  to  design  a  two-dimension¬ 
al  display  for  the  training  of  obviously  three- 
dimensional  perceptual  skills.  A  related 
issue  is  that  of  perceptual  learning  and  breadth 
of  experience.  There  is  some  indication  that 
part  of  an  LSO's  expertise  is  acquired  by 
observing  a  wide  variety  of  aircraft  approaches 
in  different  situations.  This  suggests  that 
part  of  the  LSO's  perceptual  skills  resemble 
the  pattern  vocabularies  found  in  chess 
players  (Chase  and  Simon,  1973),  circuit 
designers  (Egan  and  Schwartz,  1979),  and 
other  skilled  individuals.  Research  on  the 
structure  of  these  vocabularies  might  find 
direct  applications  to  this  project. 

I  have  mentioned  the  issue  of  automati- 
city  as  being  relevant  here.  Hooks  et  al . 
give  considerable  attention  to  this  issue, 
suggesting  that  there  are  some  perceptual 
components  of  the  task  which  demand  large 
attentlonal  resources  in  novices  but  become 
highly  automated  in  experts.  Laboratory 
techniques  for  investigating  automaticity 
should  be  advanced  enough  to  allow  us  to 
identify  those  task  components  and  perhaps 
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suggest  opportunities  for  part-task  training. 
The  other  and  more  Interesting  aspect  of 
this  Issue  Is  that  of  task  Integration,  an 
aspect  which  Hooks  et  al.  also  discuss  at 
some  length.  They  point  out  that  one  can 
functionally  divide  the  task  Into  four  parts. 
Two  of  these  have  to  do  with  assessment  of 
overall  recovery  conditions  (e.g.,  weather) 
and  are  not  relevant  here.  The  other  two 
are  assessing  the  aircraft  approach  (Stage  1) 
and  directing  the  pilot's  actions  (Stage  2). 

It  Is  not  known  If  this  Stage-1,  Stage-2 
functional  description  Is  psychologically 
valid  or  if.  In  the  expert,  the  perceptual, 
Stage-1,  processes  map  directly  to  the  action, 
Stage-2,  processes.  And,  even  If  the  two 
stages  are  functionally  separate,  they  must 
proceed  in  parallel  In  real  time,  raising 
questions  of  Interstage  Interactions,  contin¬ 
gencies,  and  competition  for  attentlonal 
resources.  I  feel  some  obligation  to  cite  the 
large  body  of  current  research  relevant  to 
this  topic,  but  fortunately  I  can  avoid  this 
chore  by  referring  you  to  Chatfleld  et  al.'s 
(1979)  thorough  review  of  the  subject. 

Decision  making  Is  also  mentioned  in 
Hooks  et  al .  as  an  important  part  of  the  LSO's 
job.  He  must  be  able  to  estimate  the  chances 
of  a  successful  landing  given  the  particular 
recovery  conditions  and  the  approach  itself. 
This,  combined  with  operational  requirements 
at  the  time,  result  in  a  decision  to  allow  a 
landing  or  wave  the  aircraft  off.  As  Hooks 
et  al .  point  out,  the  LSO  must  be  able  to 
trade  safety  for  rate  of  landing,  but  clearly 
the  tradeoff  point  is  not  set  during  the 
approach.  I  would  assume  that  in  most  cases 
the  LSO's  assessment  of  recovery  conditions 
and  operational  requirements  serve  to 
establish  a  context  for  his  handling  of  the 
approach.  This  suggests  that  some  attention 
to  the  structure  of  the  context  space  could 
benefit  the  training  of  an  LSO's  decision¬ 
making  skills.  The  ability  to  vary  approach 
characteristics  and  contextual  conditions 
independently  in  this  training  system  (again 
note  the  importance  of  control )  offers  a 
prime  opportunity  for  mapping  this  structure. 

Two  final  issues  which  I  will  discuss 
are  not  mentioned  in  Hooks  et  al.,  but 
judging  from  our  own  work,  they  should  be 
considered  relevant  here.  One  of  these  is 
the  development  of  action  skills  which 
directly  control  the  LSO's  words  and  actions. 
Some  of  our  recent  research  (Norman,  1979) 
indicates  that  we  should  be  concerned  about 
human  errors  or  slips  made  in  the  course  of 
highly  automatized  actions.  The  two  response 
systems  used  by  the  LSO  particularly  invite 
these  errors;  these  systems  are  the  moderately 
complex  key  system  used  to  operate  the  Fresnel 
Lense  Optical  Landing  System  and  the  extremely 
complex  speech  production  system  centered 
between  the  LSO's  ears  (perhaps  a  bit  nearer 


the  left  ear).  Many  of  these  slips  occur 
because  of  developing  automatlclty  and  the 
consequent  squeezing  of  tolerances  In  the 
cognitive  system.  That  we  could  use  training 
techniques  to  standardize  these  tolerances  Is 
an  exciting  but  not  farfetched  prospect. 

The  last  Issue  which  I  find  Interesting 
In  this  context  Is  best  Introduced  by  referring 
to  LSO  comments  cited  In  Hooks  et  al .  on 
"Qualities  that  separate  the  outstanding  LSO," 
namely,  "Overwhelming  emphasis  on  . . .  ability 
to  see  problems  early  on,  preferably,  before 
they  happen."  The  basis  of  this  ability  to 
plan  and  anticipate  must  have  multiple  sources. 
Some  of  It  simply  depends  on  the  knowledge  of 
how  planes  and  pilots  behave,  that  Is,  the 
same  type  of  knowledge  that  allows  for  an 
accurate  simulation  of  an  approach,  once  the 
parameters  are  set.  But  If  research  in  other 
areas  >f  psychology  and  cognitive  science 
(e.g.,  Schank  &  Abelson,  1977)  is  any  indica¬ 
tion,  this  knowledge  is  not  sufficient;  also 
involved  are  cognitive  structures  representing 
the  situation  which  go  beyond  physical  (and 
psychological)  laws.  -The  concept  of  a  "call 
window"  is  probably  one  component  of  such  a 
structure.  Such  structures  might  allow  a 
skilled  LSO  to  divide  an  approach  into  function¬ 
al  subparts,  to  detect  and  classify  aberrant 
approaches,  to  limit  searches  for  possible 
futures,  etc.  The  cutting  edge  of  our  research 
program  is  the  creation  of  methods  for  building 
explicit  computational  models  of  such  cognitive 
structures  In  the  hope  that  they  can  be  of  use 
in  training  systems  such  as  this.  These  models 
provide  much  better  ways  of  establishing  pro¬ 
ficiency  through  verbal  explanation;  they  also 
provide  a  psychological  basis  for  part-task 
training,  for  instructional  sequencing,  and 
for  proficiency  assessment.  Again,  I  can  refer 
you  to  Chatfield  et  al .  (1979)  for  further 
speculations  on  this  prospect. 

Having  finished  the  interesting  part  of 
my  talk,  I  must  now  return  to  the  question 
posed  at  its  outset,  "What  is  ONR's  interest 
in  this  training  system,  especially  consider¬ 
ing  the  long-range  nature  of  our  interests?" 

Some  of  ONR's  basic  research  is  like  baking 
a  cake,  which  one  puts  in  the  oven  and  protects 
from  all  disturbances  until  done.  However, 
the  research  represented  here  is  more  like 
cooking  crepes;  you  know  that  the  first  few 
will  not  turn  out  so  well,  but  unless  you 
start  nibbling  at  them,  the  later  ones  will 
be  just  as  bad.  Thus,  this  meeting  is  part 
of  an  ongoing  interaction  of  ONR's  basic 
research  activities  with  the  applied  research 
community,  and  we  need  to  ask  about  the 
immediate  as  well  as  the  long-term  goals  of 
this  interaction. 

Turning  to  immediate  benefits,  ONR  has 
been  involved  in  computer-based  training  and 
instruction  since  the  sixties  and  in  fact  may 


lay  claim  to  the  Invention  of  that  field. 

So  It  would  surprise  me  If  our  collective 
program  has  nothing  at  all  to  say  about 
this  particular  application.  Since  I  know 
little  about  computer-based  training  and 
less  about  LSO's,  I  have  no  suggestions  to 
make  now,  and  I  suspect  that  most  of  us  are 
in  a  similar  position.  After  the  meeting, 

I  hope  that  things  will  be  different,  and  I 
expect  that  our  contractors  can  make  two  sorts 
of  immediate  contributions.  (Both  of  these 
ideas,  by  the  way,  have  come  from  our  con¬ 
tractors,  not  myself.)  First,  there  Is  a 
good  deal  of  practical  information,  lab  lore, 
that  never  finds  its  way  Into  technical 
reports,  much  less  into  the  published  litera¬ 
ture.  Such  lab  lore  may  include  relatively 
simple  items,  such  as  solutions  to  problems 
of  contact  bounce  and  stutter  in  keyboards, 
as  well  as  high  level  information  on,  say, 
the  feasibility  of  natural  language  inter¬ 
faces.  We  rarely  publish  reports  on  our 
failures,  but  information  on  what  cannot 
be  done  is  crucially  important  in  the  applied 
world.  The  second  suggestion  for  immediate 
use  of  basic  research  is  the  use  of  existing 
theory  to  identify  important  problems,  points, 
issues,  etc.  On  this  model  we  assume  that 
instructional  design  is  something  of  a  craft 
so  that  tasks  such  as  syllabus  construction 
cannot  be  formulated  top  down  from  the 
results  of  basic  research.  What  can  be  had 
from  basic  research  are  the  types  of  results 
contained  in  Hooks  et  al .  and  Chatfield  et  al  . 
For  example,  concurrent  task  situations  can 
be  identified  so  that  performance  assessment 
can  include  dual-task  measurements.  Aspects 
of  proficient  performance  can  be  analyzed  in 
terms  of  levels  of  cognitive  control,  etc. 

I  hope  that  at  this  meeting  and  in  the 
coming  months  we  can  start  an  interaction 
with  the  system  developers  to  make  this 
unwritten  knowledge  available  and  applicable 
to  their  task. 

As  you  may  have  guessed  from  my  previous 
remarks,  there  are  considerable  points  at  which 
further  research  could  lead  to  enhanced  use 
of  this  and  similar  systems,  and  provide 
important  information  on  skill  acquisition 
and  training  technology  in  general.  Effort 
and  time  will  be  required  to  do  a  proper  job 
on  this  research,  and  most  of  it,  I  imagine, 
will  not  be  accomplished  in  time  to  influence 
the  LSO  training  project  itself.  It  is 
because  we  feel  that  the  future  Navy  is  going 
to  rely  heavily  on  these  kinds  of  training 
systems  that  we  are  prepared  to  start  research 
on  today's  problems  in  this  area.  We  will 
invest  whatever  is  appropriate  (in  the 
light  of  other  priorities)  to  provide  the 
basic  research  which  supports  the  development 
of  these  sophisticated  devices  in  general. 

(In  this  area,  as  in  most,  we  do  not  solicit 
proposals.  Rather,  the  ideas  must  come  from 
the  research  community  themselves,  and 
therefore  an  unsolicited  proposal  is  the 


most  appropriate  route.)  I  know  of  no  other 
way  of  Introducing  the  research  community  to 
the  general  Issues  connected  with  these 
systems  than  to  educate  them  In  the  particulars 
of  at  least  one  example.  Vou  should,  however, 
be  aware  that  our  Interests  are  not  primarily 
with  this  particular  training  system.  Rather, 
they  are  concerned  with  Instructional  manage¬ 
ment  for  the  roughly  defined  family  of 
systems  of  which  this  Is  one  good  instance. 
Those  who  are  Interested  will  try  to  extract 
the  general  characteristics  of  the  family, 
educate  themselves  about  other  instances, 
and,  we  hope,  be  able  to  provide  useful 
research  on  their  design  and  use. 
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THE  NAVY  PLAN  FOR  LSO  TRAINING 
AND  THE  L.OO  PHAGE  ONE  SCHOOL 


l.CDR  W.K.GRUVER 
LSO  THAI N I NG  MODEL  MANAGER 


SUMMARY 

E'.'t ablishment  of  the  LSO  Training 
M  -  it- 1  Manug-r  a'  LSO  school  has  created 
a  .•••■‘t.tral  i  zed  ••o>,*..roi  point  for  the 
management  of  ail  LSO  matters.  Person¬ 
nel  and  equipment,  are  two  of  the  major 
areas  of  concern  of  the  Model  Manager. 
Knowing  what  billets  exist,  the  people, 
their  qualifications  and  rotation  dates 
will  new  provide  better  management  to 
both  the  Navy  and  the  individual. 

Evaluation  of  existing  and  future 
equipment  is  needed  in  order  to  provide 
the  best  means  to  safely  control  air¬ 
craft  landings  aboard  aircraft  car¬ 
riers.  To  have  a  unified  community 
and  a  central  point  of  contact  in  the 
form  of  the  Model  Manager  will  aid 
both  Navy  commands  and  contractors. 

Formal  training  given  at  LSO 
School  has  increased  the  expertise 
ar.d  knowledge  level  of  all  LSOs . 
Standardization  is  now  consistent 
throughout  the  LSO  community.  Bec¬ 
ause  of  its  single  location,  infor¬ 
mation  received  daily  can  be  sent  to 
all  type  command  LSOs  explaining  the 
latest  procedures. 

BACKGROUND 

Historically  LSOs  were  trained 
on  the  job  with  very  little,  if  any, 
formal  training.  Being  an  LSO  was 
merely  a  collateral  duty  with  no 
career  potential.  With  the  evolve¬ 
nt-  nt  to  our  present  day  all-weather 
day/night  carrier  operations  and  the 


sophistication  of  today's  aircraft, 
the  role  of  the  LSO  lias  had  to  change. 
He  has  replaced  Ills  paddies  with  a 
pickle,  the  straight  deck  has  changed 
to  an  angle  deck,  and  the  cut  given  a 
prop  with  the  automatic  carrier  land¬ 
ing  of  a  jet. 

In  197^  LSO  training  was  formal¬ 
ized  into  a  single  school  located  at 
NAS  Pensacola,  Florida.  In  order  to 
be  designated  an  LSO  attendence  at 
this  school  is  required.  Sixty  hours 
of  classroom  lectures  are  given  and 
trips  are  taken  to  observe  field  equip¬ 
ment  and  day/night  carrier  operations 
aboard  ship.  Actual  LSO  equipment 
is  used  as  training  aids  and  guest 
speakers  are  scheduled  for  each  class. 

In  1979  the  Officer-in-Charge  of 
LSO  School  was  assigned  as  the  Train¬ 
ing  Model  Manager.  It  was  recognized 
that  all  of  the  LSO  billets  had  not 
been  identified  Navywide.  Also  there 
was  no  complete  list  of  people  and 
their  qualifications.  To  overcome 
this  situation  a  computer  program  was 
developed  to  monitor  all  LSOs.  Their 
location,  qualifications,  and  rotation 
dates  are  now  being  monitored  by  the 
Model  Manager  and  this  information  is 
being  sent  to  type  command  LSOs. 

A  formal  career  pattern  has  been 
established  for  LSOs.  There  is  now  a 
specific  pattern  to  follow  and  each 
stage  and  designation  has  been  defined. 
Studies  show  that  LSOs  have  a  higher 
selection  rate  to  Commander  than 
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standard  aviators. 

Becam  o  of  mir  peace  time  cotn- 
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OBJECTIVES  AND  PLANS  OF  THE  LSO  TRAINING 
R&D  PROGRAM 

OR.  R.  BREAUX 
Human  Factors  Laboratory 
Naval  Training  Equipment  Center 


INTRODUCTION 

The  major  objective  of  the  LSO  R&D  pro¬ 
gram  has  been  to  reduce  the  dependence  upon 
traditional  on-the-job-training  (OJT)  for  the 
LSO.  The  emphasis  has  not  been  to  eliminate 
OJT,  but  to  reduce  the  amount  required.  The 
two  major  design  goals  have  been  to  maintain 
proficiency  and  teach  younger  LSO’s  their  job. 
As  an  effort  to  teach,  the  R&D  program  has 
sought  definition  of  behavioral  objectives  and 
development  of  a  training  syllabus.  Systems 
have  been  explored  which  would  allow  the 
trainee  to  get  the  pickle  sooner,  because  the 
kind  of  training  that  would  be  given  would 
result  in  a  trainee  in  which  the  controlling 
LSO  has  more  confidence.  The  trainee  would  be 
responsible  for  his  own  training  and  would 
know  exactly  what  it  was  he  wants  the  LSO  to 
teach  him. 

As  an  effort  to  maintain  proficiency,  the 
R&D  program  has  explored  just  what  it  is  an 
LSO  can  see.  How  large  a  deviation  is  dectect- 
able  by  the  LSO's  eye?  What  field  of  view  is 
required  to  provide  sufficient  information  for 
control  of  the  aircraft  and  for  feedback  on 
performance?  What  are  the  ways  which  LSO  per¬ 
formance  can  be  measured?  Studies  have  been 
conducted  in  each  of  these  areas,  and  the 
results  will  be  presented  in  later  papers. 

Let's  look  at  some  specifics  in  the 
teaching  efforts.  The  three  major  models  of 
an  LSO  waving  training  system  include  waving 
itself,  aircraft/pilot/environment,  and  the 
instructor.  The  first  model,  waving,  must  be 
able  to  provide  decisions  as  to  what  call  to 
issue  given  a  particular  situation.  Thus,  in 
a  training  system  allowing  a  dynamic  inter¬ 
action  between  LSO  and  simulated  pilot,  the 
waving  model  "knows"  the  proper  action  to  take 
a*,  any  point  in  time  and  serves  as  a  basis  of 
performance  assessment.  The  second  model, 
aircraft/pilot/environment,  must  be  able  to 
provide  flight  situations  which  can  teach  a 
concept  to  the  LSO.  Thus,  as  the  trainee 
gains  in  skill  level  on  bqsic  decisions  and 
simple  situations,  the  aircraft/pilot/environ¬ 
ment  model  is  tapped  to  provide  training  sit¬ 
uations  graduated  in  complexity  and  amount  of 
information  processing  required  of  the  train¬ 
ee.  The  third  model,  instructor,  must  be  able  to 
manage  the  training  process  and  interact  with 
the  human  instructor  and  trainee.  Thus,  the 
instructor  model  analyses  performance,  cri¬ 
tiques  performance,  and  provides  objective 
recommendations  to  the  human  instructor  in 


terms  of  the  trainee's  skill  level.  So  the 
teaching  system  is  a  combination  of  subsystems 
integrated  for  the  purpose  of  assisting  the 
Instructor  LSO  to  analyze  and  interpret  train¬ 
ee  behavior,  technique,  and  skill,  and  to  aid 
the  instructor  in  developing  the  trainee  via  a 
structured  series  of  example  situations  and 
key  concepts. 

Of  course  the  teaching  effort  requires  a 
visual  system.  The  R&D  effort  has  sought  to 
determine  requirements  for  field  of  view,  re¬ 
solution  of  image,  detectable  deviations  and 
depiction  of  deviations.  Studies  will  be  re¬ 
ported  in  later  papers  which  discuss  field  of 
view  required  by  the  LSO  so  he  can  gain  in¬ 
sight  in  pilot  performance.  The  LSO  typically 
will  mentally  backtrack  from  the  wire  caught 
in  order  to  grade  the  approach.  What  require¬ 
ment  does  this  add  to  the  training  system  for 
an  LSO?  Also,  the  LSO  looks  at  a  few  lights 
at  night  to  gather  his  information.  Deviations 
from  optimal  approach  profile  must  be  detected 
and  action  taken  based  on  these  lights.  What 
type  and  size  of  deviations  car.  the  LSO  per¬ 
ceive,  and  how  consistently  do  LSO's  respond 
to  these  deviations.  Finally,  systems  must  be 
constructed  to  adequately  represent  a  visual 
scene  in  order  that  the  LSO  may  learn  his  key 
decisions  and  concepts.  What  exists  in  the 
market  place  to  meet  the  training  requirements 
in  a  cost  effective  and  training  effective  way? 

FUTURE  DIRECTIONS 

There  are  still  questions  to  be  answered, 
as  you  might  guess.  These  concern  some  very 
real  issues  which  often  determine  whether  a 
training  effort  will  eventually  be  funded. 

What  is  the  cost  effectiveness  of  a  universal 
LSO  waving  training  system?  Certainly  the 
saving  of  a  single  fatality  would  pay  for  any 
amount  of  R&D,  but  hard  numbers  are  required 
on  most  budget  justification  documents. 

How  long  would  it  take  to  pay  for  a  train¬ 
ing  system?  Estimates  are  needed  which  show 
cost  of  current  training,  effectiveness  of 
that  training,  and  skills  which  would  be  im¬ 
proved  given  application  of  principles  which 
have  been  learned  from  the  R&D  efforts.  Cer¬ 
tain  skills  must  be  selected  which  can  be 
taught  in  a  training  system,  then  ar  estimate 
must  be  made  of  the  cost  to  teach  those  skills 
vs.  things  remaining  the  way  they  are.  This 
pay-off  should  be  in  hard  numbers  as  much  as 
possible. 


Some  current  efforts  are  seeking  to  un¬ 
cover  these  numbers.  One  seeks  to  build  a 
laboratory  system  to  teach  waving  decision 
making.  The  study  will  provide  some  training 
time  estimates  for  teaching  ISO  concepts.  It 
will  be  discussed  in  detail  In  a  later  paper. 
Another  study  seeks  to  evaluate  the  LSO  Re¬ 
verse  Display,  which  will  provide  estimates 
of  the  effectiveness  of  that  approach  and 
technology.  But  future  studies  will  be  needed 
to  provide  pay-offs  in  hard  numbers  as  to 
training  savings  and  fleet  readiness  through 
the  use  of  an  LSO  training  system.  These  type 
studies  can  partially  be  done  through  analysis 
of  existing  accident  reports,  for  example,  but 
also  will  require  that  the  Navy  build  a  proto¬ 
type  system  for  field  evaluation.  Whether 
that  prototype  is  built  is  dependent  upon  the 
Fleet  itself.  It  will  be  built  and  tested 
only  if  the  LSO  community  requests  it. 


THE  LANDING  SIGNAL  OFFICER  (LSO) 

A  HISTORICAL  PERSPECTIVE 

MR.  GAIL  J.  BORDEN 

Human  Performance  Research,  Incorporated 
Goleta,  California 


The  purpose  of  my  presentation  is  to 
provide  a  historical  perspective  of  the  LSO 
and  his  role  in  the  carrier  landing  process. 

I  will  describe  the  job  of  the  LSO,  how 
changes  in  carrier  operations  have  changed  It 
over  the  years,  problems  encountered  by  the 
LSO  community  as  a  result  of  the  changes,  and 
some  of  the  R&D  studies  conducted  to  resolve 
these  problems. 

The  role  of  the  LSO  in  the  carrier 
landing  process  Is  to  provide  corrective 
assistance  to  the  pilot  during  the  final 
approach  and  landing  phase.  The  key  word  is 
assistance.  Primary  responsibility  for  air¬ 
craft  performance  resides  with  the  pilot. 

The  LSO  has  not  always  had  this  second¬ 
ary  role.  Prior  to  the  mid-1950s,  the  LSO 
directed  each  approach  and  landing  by  signal¬ 
ling  the  pilot  with  his  paddles.  He  liter¬ 
ally  flew  the  approach  for  the  pilot.  But  a 
significant  event  altered  the  LSO's  role  — 
the  paddles  were  replaced  by  a  visual  landing 
aid . 

The  change  in  the  LSO's  role  took  place 
in  1955  with  the  installation  of  the  first 
visual  landing  aid.  The  advent  of  the  jet 
aircraft  had  brought  with  it  a  requirement 
for  a  new  method  of  providing  landing  assist¬ 
ance  to  the  pilot.  The  prop  pilot  had  ample 
time  to  watch  the  LSO,  respond  appropriate¬ 
ly  to  the  paddle  signals,  and  receive  his 
"cut"  or  "wave-off"  --  but  the  jet  pilot  did 
not. 

First,  the  increase  in  approach  speeds 
of  jet  aircraft  required  a  concomitant  in¬ 
crease  in  the  length  of  the  approach  to 
landing.  It  became  obvious  to  carrier  per¬ 
sonnel  that  as  the  approach  distance  contin¬ 
ued  to  lengthen  with  increases  in  approach 
speeds,  it  would  no  longer  be  possible  for 
pilots  to  see  the  signals  given  by  the  LSO 
with  his  paddles.  A  system  was  required  that 
would  provide  glide-slope  information  to  the 
pilot  at  distances  of  one  mile  or  more  from 
the  ship. 

Second,  the  flat  approach  formerly 
flown  by  the  prop  pilot  to  the  vicinity  of 
the  ramp,  followed  by  either  a  cut  or  wave- 
off  signal  from  the  LSO,  was  not  suitable  to 
the  flight  characteristics  of  jet  aircraft. 

A  flat  approach  can  result  in  either  a 
"bolter"  (the  aircraft's  hook  fails  to  en¬ 
gage  one  of  the  arrestment  wires  and  the 
pilot  reenters  the  traffic  and  makes  another 


approach)  or  a  hard  landing.  A  hard  landing 
occurs  when  sink  rate  is  excessive  as  a  re¬ 
sult  of  the  sudden  cut  in  power.  Jet  aircraft 
require  a  constant  glide-slope  angle  approach, 
with  power-on  and  no  cut  until  the  hook  has 
engaged  the  wire.  This  means  that  glide- 
slope  information  has  to  be  provided  to  the 
pilot  continuously  to  touchdown,  well  past 
the  point  that  the  LSO  would  be  in  view.  For 
these  reasons,  and  others,  the  visual  landing 
aid  system  was  adopted. 

The  primary  role  originally  played  by 
the  LSO  was  reduced  to  a  secondary  role:  the 
"paddles"  LSO  would  tell  the  pilot  when  and 
what  to  correct.  In  the  visual  landing  aid 
approach,  it  is  first  up  to  the  pilot  to 
analyze  the  approach  and  effect  a  solution. 

If  approach  performance  becomes  unacceptable, 
the  LSO  will  offer  corrective  assistance. 

When  the  visual  landing  aid  was  intro¬ 
duced,  and  accompanied  to  introduction  of  the 
angle  deck,  many  believed  that  the  LSO  had 
become  obsolete  and  eventually  would  be  re¬ 
moved  from  the  landing  process  altogether  as 
pilots  became  proficient  in  visual  landing 
aid  techniques.  Others  believed  that  there 
would  always  be  a  requirement  for  someone  to 
monitor  aircraft  recoveries,  and  they  suggest¬ 
ed  that  the  LSO  continue  on  in  that  role  with 
the  title  Landing  Safety  Officer. 

This  situation  existed  for  several  years 
The  carrier  landing  accident  rate  decreased 
significantly  from  the  period  before  the  intro 
duction  of  visual  landing  aids  in  1954  to  the 
period  after  introduction  was  largely  com¬ 
pleted  in  1957:  the  rate  decreased  from  35 
accidents  per  10,000  landings  in  1954  to  nine 
accidents  per  10,000  landings  in  1957  --  a 
four-fold  improvement  in  just  four  years. 

This  dramatic  reduction  in  the  accident  rate 
seemed  to  confirm  the  judgement  that  the  LSO 
would  soon  be  either  completely  supplanted 
by  the  visual  landing  aid  or  reduced  to  the 
role  of  safety  observer. 

Then  a  number  of  events  took  place  that 
eventually  were  to  regain  for  the  LSO  the  im¬ 
portance  and  status  he  had  previously  been 
accorded.  The  reduction  in  the  accident  rate 
began  to  level  off.  Although  the  accident 
rate  slowly  continued  to  be  reduced,  it  re¬ 
mained  substantially  higher  than  the  accident 
rate  at  conventional  airfields.  The  cost  of 
accidents  began  to  rise  because  of  the  in¬ 
creasing  cost  of  aircraft,  other  equipment, 
and  personnel  .  The  percentage  of  hard  land- 


Ings  and  ramp  strikes  Increased  from  approxi¬ 
mately  40%  of  total  carrier  landing  accidents 
In  1954  to  80%  In  1964.  The  tempo  and  comp¬ 
lexity  of  carrier  air  operations  Increased 
and  operations  were  extended  Into  more  severe 
environments,  night  and  all-weather  operations 
Increased,  heavier  and  faster  aircraft  were 
Introduced,  and  the  number  of  different  types 
of  aircraft  Increased.  Not  only  were  there 
more  different  types  of  aircraft,  but  they 
varied  markedly  in  approach  and  landing 
flight  characteristics.  These  changes  criti¬ 
cally  reduced  the  margin  for  error  in  carrier 
approaches . 

The  results  of  these  changes  was  a 
reassessment  of  the  requirements  for  the  safe 
and  efficient  recovery  of  carrier  aircraft. 

If  the  accident  rate  was  to  be  further  re¬ 
duced  while  the  tempo  and  complexity  of  air 
operations  increased,  pilots  would  require 
more  assistance  in  the  approach  than  could  be 
provided  by  the  visual  landing  aid.  Passive 
information,  by  itself,  was  often  not  enough 
for  successful  recovery  of  aircraft  at  night, 
under  conditions  of  limited  visibilities  and 
pitching  decks.  Active  LSO  assistance,  not 
simply  monitoring,  was  essential  for  most 
pilots  under  these  conditions.  In  the  British 
Navy,  the  LSO  had  actually  been  removed  from 
the  carrier  landing  process  after  introduction 
of  the  visual  landing  aid  but  was  placed  back 
in  the  process  after  the  accident  rate  began 
to  increase. 

The  LSOs  regained  the  important  role 
that  they  had  previously  held  in  the  carrier 
landing  process.  However,  it  was  not  regained 
without  difficulty.  The  general  downgrading 
of  the  visual  landing  aid  was  accompanied  by 
a  diminishment  in  the  status  that  had  been 
previously  accorded  the  job.  The  number  of 
LSOs  dwindled,  command  attention  waned,  and 
LSO  training  and  quota  requirements  were  not 
enforced.  The  job  no  longer  had  the  same 
appeal  to  pilots  now  that  much  of  the  prestige 
was  removed.  The  result  was  a  significant  de¬ 
crease  in  the  number  of  volunteers  for  LSO 
training.  Not  only  was  the  quantity  of  LSOs 
affected,  but  also  the  quality.  The  billet  no 
longer  appealed  to  ambitious,  career-oriented 
junior  officers.  They  viewed  the  billet  as 
being  detrimental  to  their  career  and  pre¬ 
ferred  to  serve  in  more  "career  enhancing" 
billets . 

Although  the  LSO  billet  regained  much  of 
the  importance  it  had  lost  earlier,  status  was 
regained  more  slowly.  Several  years  passed 
before  a  nucleus  of  LSOs  was  formed  with  the 
personality  traits  and  skills  that  once  again 
could  command  the  respect  of  squadron  pilots. 
However,  the  attitude  persisted  that  the 
billet  was  not  career  enhancing,  and  the  LSO 
community  was  unable  to  attract  enough  quali¬ 
fied  junior  officers  to  fill  all  of  the  LSO 


billets . 

The  problem  of  attracting  junior  officers 
was  compounded  by  the  demands  of  the  billet. 

The  increased  complexity  and  tempo  of  carrier 
air  operations  caused  a  concomitant  increase 
in  the  demands  of  the  job.  An  appreciation  of 
these  demands  can  be  gained  by  examining  the 
unique  aspects  of  the  LSO  billet  that  set  it 
apart  from  other  billets  in  which  naval  aviators 
can  serve.  It  requires  a  longer  training  per¬ 
iod.  The  average  length  of  training  to  be  able 
to  control  a  single  type  of  aircraft  during 
both  day  and  night  recovery  conditions  is 
approximately  15  months  —  nearly  as  long  as 
flight  training  itself.  After  this  initial 
qualification,  it  takes  approximately  two  more 
years  before  an  LSO  is  qualified  to  control 
all  types  of  aircraft  that  comprise  a  carrier 
air  wing,  and  only  a  few  LSOs  are  capable  of 
achieving  this  qualification.  Once  trained, 

LSOs  remain  in  the  billet  longer  than  their 
non-LSO  contemporaries  remain  in  their  respec¬ 
tive  billets.  The  average  length  of  time 
spent  in  the  LSO  billet,  after  training,  is  six 
years.  LSOs  tend  to  work  long  and  awkward 
hours:  13  to  14  hour  work  days  are  not  un¬ 
common,  and  much  of  this  time  occurs  in  the 
late  evenings  and  early  mornings.  These  hours 
are  not  confined  to  shipboard  deployments. 

When  ashore,  LSOs  are  responsible  for  the 
carrier  landing  training  of  pilots.  Much  of 
this  training  takes  place  at  night  and  at  sea. 

It  is  not  uncommon  for  a  training  LSO  to  spend 
one-third  of  his  shore-based  tour  away  from 
home.  Serving  in  the  LSO  billet  both  during 
training  and  after  qualification  does  not  re¬ 
duce  in  any  way  the  LSO's  obligations  and 
responsibilities  as  a  pilot.  LSOs  once  were 
assigned  to  carriers  as  part  of  ship's  company 
in  non-flying  status.  Today,  they  are  active 
members  of  the  air  wing,  participating  in  all 
squadron  flight  operations,  in  addition  to 
serving  as  squadron  or  wing  LSOs. 

The  demands  of  the  LSO  billet  and  the 
attitude  that  the  billet  was  non-career  en¬ 
hancing  combined  to  produce  serious  personnel 
deficiencies  in  the  LSO  billet.  These  problems, 
and  others,  were  identified  in  a  study  I  per¬ 
formed  in  1967  for  the  Office  of  Naval  Re¬ 
search.  The  purpose  of  the  study  was  to 
identify  problems  that  LSOs  were  encountering 
in  their  job  that  if  eliminated  would  enhance 
their  performance  and  the  overall  safety  and 
efficiency  of  carrier  air  operations. 

Interviews  conducted  with  LSOs  during 
the  initial  phase  of  the  study  revealed  that 
they  were  most  concerned  about  personnel  pro¬ 
blems.  It  was  revealed  that  the  demands  of 
the  job  and  the  non-career  enhancing  attitude 
was  not  only  detrimentally  affecting  the  re¬ 
cruitment  of  junior  officers  as  trainees,  but 
also  the  retention  of  senior  LSOs.  They  were 
having  to  devote  more  hours  to  the  job  and  more 


time  in  the  billet  to  make  up  for  the  short¬ 
ages  in  qualified  LSOs .  They  felt  this  was 
detrimentally  affecting  their  chance  for  pro¬ 
motion  to  the  ranks  of  senior  officer  and 
selection  for  squadron  command.  They  reason¬ 
ed  that  the  extensive  training  period,  the 
length  of  time  they  served  in  the  billets, 
and  the  demands  of  the  job,  reduced  their 
opportunity  to  serve  in  the  variety  of 
squadron  and  wing  billets  that  their  non-LSO 
contemporaries  serve  in  during  their  careers; 
and  that  this  caused  them  to  lack  the  general¬ 
ized  background  considered  essential  by  the 
Navy  for  optimum  career  development.  As  a 
result,  they  felt  that  they  did  not  have  the 
same  opportunity  as  their  non-LSO  contempor¬ 
aries  for  promotion  to  senior  ranks  and  sel¬ 
ection  for  command.  Furthermore,  they  were 
able  to  substantiate  this  claim  by  citing 
examples  of  senior  LSOs  who  were  not  promoted 
or  screened. 

As  part  of  the  ONR  study,  I  obtained 
data  from  the  Bureau  of  Personnel  that  indi¬ 
cated  that  there  was  no  justification  for  this 
attitude.  In  the  most  recent  year  group  con¬ 
sidered  for  promotion  and  selection  for 
command  at  the  time  of  the  study,  the  LSOs 
were  significantly  more  successful  than  their 
non-LSO  contemporaries .  Indeed,  it  appeared 
that  serving  in  the  LSO  billet  enhanced  their 
chance  for  promotion  and  selection  for  com¬ 
mand.  I  believe  that  it  was  true  at  one  time 
that  LSOs  were  not  as  successful  as  their  non- 
LSO  contemporaries.  However,  I  believe  that 
this  was  the  result  of  the  officers  that  were 
attracted  to  the  billet  during  the  years 
following  the  introduction  of  visual  landing 
aids,  rather  than  their  association  with  the 
billet.  These  officers  lacked  the  attributes 
that  the  Navy  considers  important  for  pro¬ 
motion  and  selection  for  command,  and  there¬ 
fore,  were  less  successful  than  their  non-LSO 
contemporaries.  Recent  data  obtained  from 
Bupers  indicates  that  LSOs  continue  to  be 
more  successful  than  their  non-LSO  contempor¬ 
aries  in  both  promotion  and  selection  for 
command . 

The  publication  of  these  findings  dis¬ 
pelled  the  attitude  that  the  billet  was  not 
career  enhancing.  Recruitment  and  retention 
began  to  improve.  Also,  as  part  of  the  ONR 
study,  I  worked  with  Bupers  to  develop  an 
LSO  career  assignment  policy  that  further 
enhanced  LSO  recruitment  and  retention.  This 
policy  is  still  in  use  today.  To  my  know¬ 
ledge,  this  is  the  first  time  in  recent 
history  that  Bupers  has  established  a  special 
assignment  policy  within  the  unrestricted 
line  officer  category. 

Another  problem  Identified  In  the  ONR 
problem  analysis  study  was  deficiencies  in 
the  design  of  the  LSO  work  station  or  platform. 
Deficiencies  exist  in  the  design,  location. 


arrangement,  and  integration  of  displays  and 
equipments  intended  to  assist  the  LSO  in  con¬ 
trolling  the  aircraft.  A  major  problem  is 
the  location  of  displays  of  aircraft  perform¬ 
ance  such  as  speed,  glide-slope  and  line-up 
discrepancies,  range,  and  so  on.  These  dis¬ 
plays,  and  others,  are  located  in  a  console 
at  the  LSOs'  feet  and  slightly  to  his  right, 
while  the  aircraft  is  approaching  from  the 
left  and  slightly  above  eye  level.  The  LSO 
simply  cannot  take  his  eyes  off  the  approach¬ 
ing  aircraft  to  look  at  the  displays,  partic¬ 
ularly  in  the  critical  terminal  phase  of  the 
approach.  As  a  result,  the  utility  of  the 
displays  is  severely  limited. 

Gross  location  is  not  the  only  problem. 
Displays  are  also  not  optimally  integrated 
for  use  by  the  LSO.  Individual  displays  are 
positioned  on  the  console  without  considera¬ 
tion  to  their  priority  of  importance  of  the 
information  displayed.  Also,  the  design  of 
individual  displays  is  poor,  which  further 
affects  their  utility.  Communication  equip¬ 
ment  is  also  inadequately  designed  and  located 
And  the  utility  of  all  the  displays  and  equip¬ 
ments  is  seriously  affected  by  excessive  down 
time. 

At  the  completion  of  the  problem  analy¬ 
sis  study,  ONR  funded  a  follow-on  study  to 
redesign  the  LSO  platform  consistent  with 
present  and  future  LSO  task  requirements. 

The  major  result  of  this  study  was  the  design 
of  a  new  LSO  display  console.  The  console 
was  designed  to  satisfy  certain  fundamental 
requirements.  First,  it  relocates  displays 
more  in  line  with  the  approaching  aircraft. 

The  console  is  mounted  on  a  stand  that  can  be 
elevated  to  accommodate  the  height  of  individ¬ 
ual  LSOs  and  aligned  with  the  path  of  approach 
ing  aircraft.  Relocation  alone,  however,  was 
not  enough  to  provide  maximum  utilization  of 
the  displays.  Relocation  would  improve 
utility  but  still  would  require  the  LSO  to 
take  his  eyes  off  the  aircraft  to  read  the 
displays  because  of  having  to  change  focus. 

A  second  requirement  then  was  to  adopt  a 
display  format  that  would  eliminate  the  need 
for  changes  in  accommodation.  This  was 
accomplished  by  adopting  a  head-up  display 
(HUD)  format.  Information  presented  on  the 
HUD  appears  to  be  superimposed  on  the  LSOs' 
forward  field  of  view.  Thus,  changes  in 
accommodation  to  monitor  the  displays  are  not 
required . 

A  third  requirement  was  to  improve  the 
LSOs'  ability  to  control  aircraft  in  minimum 
visibility  conditions  when  he  cannot  see  the 
aircraft  until  it  breaks  out  in  the  terminal 
phase  of  the  approach.  This  was  accomplished 
by  providing  the  LSO  with  an  Integrated, 
sensor  driven  display  of  line-up  and  glide- 
slope  information  on  the  HUD  as  well  as  sink- 
rate,  speed,  range,  and  deck  motion.  Dis- 


playing  this  information  on  the  HUD  allows 
LSOs  to  simultaneously  monitor  the  displays 
and  scan  the  overcast  for  the  approaching 
aircraft,  thereby  providing  the  ISO  with  air¬ 
craft  trend  information  before  he  sees  the 
aircraft. 

The  new  console  was  field  tested  at 
Patuxent  River  in  1973,  by  approximately  50 
LSOs,  and  evaluated  at  sea  in  1975  on  the  USS 
Ranger  (CV-61).  It  is  presently  installed 
on  two  carriers,  and  plans  have  been  formulat¬ 
ed  by  the  Naval  Air  Systems  Command  for 
installation  on  all  other  carriers. 

I  would  like  to  change  the  topic  of 
discussion  now  to  LSO  training.  As  previous¬ 
ly  mentioned,  the  LSO  is  responsible  for 
controlling  aircraft  approaches,  and  during 
the  final  seconds  of  the  approach,  for  making 
the  critical  final  decision  that  determines 
if  the  aircraft  can  be  safely  brought  aboard 
the  carrier.  In  this  process,  he  employs  some 
of  the  most  complex  perceptual  and  judge¬ 
mental  skills  that  naval  aviators  are  called 
upon  to  make.  His  decision  depends  upon  the 
perception  of  subtle  visual  and  auditory 
cues  which  vary  with  different  types  of  air¬ 
craft,  the  idiosyncracies  of  different 
pilots,  environmental  factors  such  as  weather, 
carrier  motion,  wind  over  deck,  air  wake  tur¬ 
bulence,  time  of  day,  operational  time  con¬ 
straints,  and  many  other  factors.  Some  idea 
of  the  complexity  of  the  job  is  indicated  by 
the  amount  of  time  required  to  become  quali¬ 
fied  as  a  squadron  and  air  wing  LSO. 

Despite  the  complexity  and  importance 
of  the  LSO's  job,  LSO  training  has  remained 
essentially  the  same  since  LSOs  were  first 
introduced  into  carrier  aviation.  No  other 
aspect  of  carrier  aviation  training  has  re¬ 
mained  as  unchanged,  and  received  so  little 
attention  from  training  specialists.  Vet, 
with  the  possible  exception  of  landing 
aboard  carriers,  no  other  task  performed  by 
naval  aviators  is  more  demanding  or  complex. 

LSO  training  today  is  performed  in  the 
same  manner  as  it  was  30  years  ago  --  inform¬ 
al,  on-the-job  training.  Trainees  observe, 
and  later  control,  approaches  under  the 
supervision  of  senior  squadron  and  air  wing 
LSOs.  Guidance  for  training  is  provided  by 
rather  vague  statements  of  requi rements , 
objectives,  and  evaluative  criteria.  The 
lack  of  specific  guidelines,  and  the  individ¬ 
ual  differences  in  the  ability  of  senior  LSOs 
to  impart  LSO  skills,  results  in  a  wide  range 
of  training  effectiveness.  O/er  the  years, 
the  LSO  community  has  recognized  the  need  to 
improve  training  and  has  made  several  attempts 
to  initiate  studies.  However,  until  recently, 
these  attempts  have  been  unsuccessful . 

The  need  to  Improve  training  received 


renewed  emphasis  several  years  ago  when  the 
carrier  deployment  schedule  was  changed.  Prior 
to  this  change,  carriers  remained  in  port  for 
approximately  6  to  8  months  between  deploy¬ 
ments.  The  change  increased  the  in-port 
period  to  approximately  13  to  15  months.  This 
means  that  carrier  pilots  and  LSOs  are  operat¬ 
ing  at  sea  for  about  half  as  long  as  they  were 
several  years  ago.  The  reduction  in  at-sea 
time  reduces  the  opportunity  for  pilots  to 
acquire  basic  carrier  landing  skills  and  once 
acquired,  to  maintain  proficiency  in  these 
skills.  LSOs,  too,  have  less  opportunity  to 
acquire  and  maintain  their  skills. 

Concern  for  the  effect  of  the  reduction 
in  at-sea  time  on  the  safety  of  carrier  opera¬ 
tions  led  CNO,  COMNAVAIRPAC,  and  COMNAVAIRLANT, 
to  convene  an  LSO  meeting  in  December  of  1973 
to  develop  recommendations  for  reducing  the 
impact  of  the  change  in  carrier  deployment 
schedules  on  landing  performance.  The  assump¬ 
tion  was  that  the  most  practical  way  of  avoid¬ 
ing  an  increase  in  the  accident  rate  was  through 
improved  LSO  performance,  because  apart  from 
the  pilot,  the  LSO  has  the  most  direct  influ¬ 
ence  on  the  safety  of  carrier  operations.  The 
attendees  felt  that  if  LSO  performance  was  to 
be  improved,  improvement  was  most  likely  to 
be  achieved  through  training.  However,  apart 
from  some  recommendations  for  changes  in  organi¬ 
zation,  they  could  not  make  any  specific 
recommendations  for  improving  training.  They 
felt  their  job  was  far  too  complex  to  be 
improved  without  assistance  from  training 
specialists.  Therefore,  attendees  recommended 
that  a  training  requirements  study  be  performed 
that  would  systematically  examine  their  job 
and  develop  recommendations  for  an  improved 
program  of  training  --a  goal  of  the  LSO 
community  long  before  the  change  in  carrier 
deployment  schedule.  This  recommendation 
was  translated  by  the  Navy  into  the  present 
NTEC  LSO  training  effort. 

The  change  in  carrier  deployment  sched¬ 
ule  created  a  renewed  emphasis  on  improving 
training  but,  at  the  same  time,  reduced  the 
opportunity  for  LSOs  to  observe  and  control 
aircraft  at  sea  --  the  most  important  aspect 
of  LSO  training.  Therefore,  emphasis  in  the 
NTEC  program  has  been  placed  on  developing  a 
training  technique  that  will  provide  a  real¬ 
istic,  alternative,  training  experience. 

Present  shore-based  training  does  not  provide 
a  realistic  alternative.  Although  an  essential 
aspect  of  LSO  training,  observation  and  control 
of  aircraft  at  shore  facilities  is  limited  in 
its  capability  to  prepare  trainees  for  at-sea 
operations.  Differences  exist  in  aircraft 
approach  performance  due  to  the  lack  of 
carrier  movement  and  reduced  wind  velocity. 

Also,  additional  visual  cues  are  provided  by 
lights,  structures,  and  surrounding  terrain 
which  makes  the  task  of  controlling  aircraft 
significantly  easier.  Therefore,  NTEC  elected 


to  develop  a  visual  simulator  that  would 
provide  a  realistic  and  effective  simulation 
of  the  carrier  recovery  approach  and  environ¬ 
ment. 

However,  the  goal  of  the  NTEC  effort  Is 
not  limited  to  providing  an  alternative  train¬ 
ing  technique  for  the  at-sea  experience.  It 
is  also  intended  to  provide  a  more  effective 
overall  training  program.  This  will  be  accom¬ 
plished  by  augmenting  the  at-sea  experience 
with  a  wider  and  more  diverse  range  of 
approaches  and  recovery  conditions  than 
trainees  typically  experience  during  carrier 
deployment.  On  carriers,  approaches  cannot 
be  allowed  to  deteriorate  to  a  point  where 
they  become  dangerous.  Thus,  trainees  seldom 
are  provided  with  the  opportunity  of  observing 
the  kinds  of  dangerous  approaches  that  they 
are  expected  to  prevent  as  qualified  LSOs . 
Also,  because  of  safety  considerations, 
senior  LSOs  are  reluctant  to  allow  trainees 
to  control  approaches  during  extreme  condi¬ 
tions  of  limited  visibility,  pitching  deck, 
aircraft  battle  damage  or  malfunction,  and 
so  on.  Simulator  training,  on  the  other 
hand,  is  not  limited  by  these  constraints. 
Trainees  can  be  provided  with  the  opportunity 
to  observe  and  control  a  wider  range  of 
approaches,  including  approaches  that  termi¬ 
nate  in  accidents,  under  the  full  range  of 
environmental  conditions. 

The  simulator  is  also  anticipated  to 
provide  an  effective  method  of  introducing 
basic  skills  to  new  trainees.  Approaches 
can  be  replayed  for  detailed  analysis  and 
immediate  performance  feedback.  Furthermore, 
simulator  training  will  not  be  limited  by 
aircraft/pilot  availability  and  weather  condi¬ 
tions  . 

The  need  to  enhance  ISO  training  has 
received  additional  emphasis  with  the  recent 
change  in  commercial  airlines  hiring  policy. 
This  change  in  policy,  and  other  factors, 
have  resulted  in  a  pilot  and  LSO  retention 
rate  of  only  27  per  cent  after  the  first 
obligated  tour.  The  result  of  this  change 
has  caused  serious  deficiencies  to  develop  in 
the  number  of  qualified  squadron  and  air  wing 
LSOs.  These  deficiencies  make  the  need  for 
improved  training  techniques  all  the  more 
important,  especially  techniques  that  will 
speed  the  acquisition  of  basic  LSO  skills. 

However,  before  an  effective  simulator 
training  technique  can  be  developed,  we  must 
learn  more  about  how  the  LSO  performs  his 
job.  In  my  next  presentation,  I  will  describe 
the  carrier  landing  process  in  more  detail, 
and  the  LSO's  role  in  that  process  in  order 
to  underscore  the  complexity  of  the  LSO's  job, 
and  the  difficulty  of  objectively  assessing 
the  job  for  the  purpose  of  developing  training 
requirements . 
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INTRODUCTION 


Human  Performance  Research  (HPR)  was 
tasked  by  NTEC  to  perform  a  systematic 
analysis  of  the  LSO's  job  of  controlling 
aircraft  in  the  final  approach  and  landing 
phase  of  carrier  recovery  operations  for  the 
purpose  of  developing  an  LSO  decision-making 
model.  Dr.  Michael  McCauley,  who  is  the 
project  director  of  this  study,  will  describe 
the  study  and  some  of  the  preliminary 
results.  However,  before  his  presentation, 

I  would  like  to  take  a  few  minutes  to 
describe  the  LSO's  job  and  the  difficulty  of 
analyzing  the  job. 

The  Complexity  of  the  LSO's  Job 

The  LSO  monitors  each  approaching 
aircraft  and,  if  he  feels  that  the  pilot 
requires  assistance,  will  transmit  by  radio 
a  corrective  action.  If  the  pilot  fails  to 
respond  or  if  the  approach  continues  to 
deteriorate,  the  LSO  can  exercise  his  right 
to  terminate  the  approach  by  commanding  a 
wave-off. 

This  is  not  an  easy  task.  Approach 
speeds  of  today's  jet  aircraft  range  between 
100  and  140  knots.  The  average  duration  of 
the  final  approach  is  30  seconds.  With  a 
boarding  interval  of  about  40  seconds  during 
day  recoveries,  and  60  seconds  at  night,  the 
LSO  must  detect  deviations  in  approach 
performance  as  soon  as  they  occur,  and  offer 
accurate,  corrective  assistance  immediately. 
He  must  do  this  each  hour  and  45  minutes  for 
approximately  20  recovering  aircraft  through¬ 
out  the  typical  12-hour  period  of  cyclic 
operations. 

The  LSO  has  two  major  responsibilities 
as  defined  in  the  C V  and  LSO  NAT0PS  manuals. 
First,  and  most  important,  he  is  responsible 
for  the  recovery  of  aircraft.  Second, 

he  is  responsible  for  the  r r  ious 
recovery  of  aircraft.  Of  these  two  responsi¬ 
bilities,  the  one  that  is  most  often 
emphasized  is  the  LSO's  responsibility  for 
the  safety  of  the  recovery.  Many  people  who 
lack  first-hand  knowledge  of  carrier  opera¬ 
tions  view  the  LSO  as  an  arbiter  of  pilot 


performance  who  monitors  the  approach 
and  issues  judgements  in  the  form  of 
wave-offs  whenever  an  approach  deviates 
from  some  acceptable  standard  of  perfor¬ 
mance.  Furthermore,  many  people  believe 
that  approach  performance  is,  for  the  most 
part,  optimum,  and  that  substandard 
approaches  are  infrequent  and  confined, 
largely,  to  inexperienced  pilots.  The 
fact  of  the  matter  is  that  the  vast 
majority  of  approaches  deviate  in  some  way 
from  optimum  performance.  Optimum  perfor¬ 
mance  is  the  exception  rather  than  the 
rule.  Deviations  occur,  and  under  severe 
weather  conditions  during  night  recoveries, 
they  can  be  quite  larg,_.  How  does  this 
affect  the  LSO?  It  brings  into  conflict 
his  major  responsibilities  for  both  the 
safe  and  expeditious  recovery  of  carrier 
aircraft.  To  expedite  the  recovery,  the 
LSO  must  often  work  closer  to  the  extremes 
of  acceptable  performance  than  he  would  if 
safety  were  his  only  consideration. 

Whether  to  allow  a  marginal  approach  to 
continue  in  the  expectation  that  it  can  be 
corrected  through  assistance,  or  to 
terminate  it  by  commanding  a  wave-off  is  a 
decision  that  requires  the  utmost  skill  and 
knowledge  of  the  LSO.  Five  aircraft 
performance  parameters  are  available  to  the 
LSO  for  judging  aircraft  approach  perfor- 
mance--glide-slope,  line-up,  power, 
attitude,  and  speed  (or  angle  of  attack). 
Approach  performance  parameters  are 
perceived  primarily  by  visual  cues, 
although  auditory  information  about 
aircraft  power  also  is  important.  The 
judgemental  process  can  be  divided  into 
two  tasks.  First,  the  LSO  must  discern 
deviations  from  optimum  performance.  This 
task  requires  the  LSO  to  judge  whether  or 
not  the  aircraft  parameters  have  deviated 
from  an  optimum  state  and  to  judge  the 
amount  of  deviation.  For  example,  the  LSO 
must  judge  whether  or  not  the  aircraft  is 
properly  lined-up.  Second,  the  LSO  must 
discern  the  rate  of  change  in  aircraft 
parameters,  regardless  of  whether  or  not 
they  are  optimum.  For  example,  the  LSO 
must  judge  whether  line-up  has  changed  or 
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is  changing  from  its  immediately  preceding 
state. 

The  LSO  judges  approach  performance  and 
offers  corrective  assistance  to  the  pilot, 
when  required,  to  correct  deviations  in 
approach  performance .  However,  there  is  a 
point  in  the  approach  beyond  which  the  LSO's 
control  over  the  aircraft  is  limited.  This 
point  is  the  result  of  a  time  lag  that  exists 
in  both  aircraft  engine  and  pilot  response. 

In  aircraft,  it  is  a  lag  in  engine  response 
time  to  changes  in  power  setting.  In  pilots, 
it  is  the  laq  between  detection  of  a  devia¬ 
tion  in  approach  performance  and  the 
initiation  of  an  effective  correction,  or 
between  the  time  that  the  LSO  calls  for 
corrective  action  and  the  pilot  initiates 
that  action. 

These  tune  lags  limit  the  corrective 
action  that  can  be  taken  to  prevent  carrier 
landing  accidents  to  a  point  well  before  the 
aircraft  crosses  the  ramp  of  the  carrier. 

If  an  aircraft  goes  below  the  glide-slope 
late  in  the  terminal  phase  of  the  approach, 
the  engine  response  laa,  pilot  response  laq, 
or  both,  may  preclude  LSO  corrective  assis¬ 
tance  from  preventing  an  accident. 

The  specific  point  in  the  approach  where 
the  LSO's  control  becomes  limited  varies  with 
aircraft  type,  pilot  proficiency,  immediately 
preceding  glidepath  performance,  and  other 
factors,  but  it  can  be  regarded  as  approxi¬ 
mately  one  or  two  seconds  before  reaching  the 
ramp  of  the  carrier.  The  LSO's  objective  is 
to  ensure  that  the  aircraft  is  established  in 
an  optimum  or  near  optimum  state  for  recovery 
before  reaching  this  point. 

The  LSO's  task  is  twofold.  He  must 
first  decide  whether  or  not  the  aircraft  is 
in  an  optimum  or  near  optimum  state.  If 
deviations  exist  and  changes  are  not  taking 
place  that  will  eliminate  these  deviations, 
he  will  issue  a  wave-off.  If  performance  is 
acceptable,  the  LSO  must  then  decide  whether 
or  not  it  willremain  acceptable  during  the 
final  one  or  two  second  period  of  the 
approach.  He  does  this  by  predicting 
performance. 

Predicting  performance  is  an  essential 
aspect  of  the  LSO's  job  and  is  not- limited 
to  the  terminal  phase  of  the  approach. 
Throughout  the  approach  the  LSO  is  using  his 
predictive  capability  to  prevent  deviations 
so  that  the  aircraft  will  arrive  at  the 
decision  wave-off  point  in  an  acceptable 
state  for  the  landing.  He  does  this  by 
iudqing  the  deviations  and  rates  of  change 
that  exist  at  a  precise  moment  in  time  and 
then  extrapolating  the  future  state  of  the 
aircraft.  The  LSO  is  continually  extra¬ 
polating  because  his  ultimate  objective  is 
to  brinq  the  aircraft  to  the  decision 


wave-off  point  with  its  various  parameters 
falling  within  very  narrow  limits.  Where 
the  aircraft  is  at  any  precise  moment  in 
time  is  almost  never  as  important  as  where 
it  will  be.  Predicting  future  parameter 
values  from  instantaneous  parameter  values 
is  undoubtedly  the  most  difficult  task 
performed  by  the  LSO.  (Borden,  1969.) 

Lack  of  recognition  of  the  limits  of 
LSO  control  has  led  people  to  believe  that 
the  LSO  should  be  able  to  prevent  all 
carrier  landing  accidents.  Unfortunately, 
this  isn't  true.  If  the  pilot  makes  an 
improper  control  action  after  passing  the 
decision  wave-off  point,  there  may  be  very 
little  the  LSO  can  do  to  prevent  an  accident 
from  occurring.  The  majority  of  accidents 
in  which  the  LSO  is  cited  as  a  causal  or 
contributing  factor  probably  do  not  occur 
because  the  LSO  failed  to  detect  a  deviation 
in  performance,  but  because  the  LSO  failed 
to  accurately  predict  approach  performance 
for  the  final  one  to  two  seconds  before 
crossing  the  ramp. 

The  purpose  of  this  description  of  the 
LSO's  job  is  to  underscore  its  complexity, 
and,  therefore,  the  difficulty  of  examining 
the  job  for  the  purpose  of  developing  a 
training  program.  The  authors'  understand¬ 
ing  of  the  LSO's  job  is  based  entirely  on 
observation  of  the  carrier  recovery  process 
and  interviews  with  LSOs.  It  lacks  the 
rigorous,  systematic  examination  required 
for  development  of  a  training  program.  We 
do  not  know,  for  example,  the  approach 
parameter  values  judged  as  acceptable 
performance  by  LSOs  and  the  effect  on  these 
values  of  such  factors  as  weather,  pilot 
proficiency,  LSO  proficiency,  carrier  deck 
motion,  and  so  on.  The  largest  source  of 
variability  may  be  the  LSO's  dual  responsi¬ 
bility  for  both  the  safe  and  expeditious 
recovery  of  aircraft.  However,  the  values 
of  acceptable  performance  and  the  factors 
that  influence  these  values  cannot  be 
determined  without  conducting  a  rigorous 
examination  of  the  LSO's  job. 

STUDY  PHASE  I 

The  study  tasks  to  be  performed  by  HPR 
were:  1)  analyze  the  decision  making  process 
of  the  Landing  Signal  Officer  (LSO)  during 
carrier  aircraft  recovery  operations;  2) 
identify  aircraft  approach  parameters  that 
are  used  by  LSOs  to  judge  approach  perfor¬ 
mance;  3)  identify  situational  and  environ¬ 
mental  variables  that  influence  the  LSO's 
task;  4)  estimate  the  LSO's  decision  criteria 
for  issuing  corrective  assistance  calls  to  the 
pilot;  5)  catalogue  and  analyze  the  standard 
and  non-standard  verbal  responses  commonly 
used  by  LSOs;  6)  suggest  candidate  perfor¬ 
mance  measures  applicable  to  an  automated  LSO 
trainer;  and  7)  synthesize  this  information 


18 


in  an  LSO  model  suitable  for  translation  to 
software  for  application  in  an  automated  LSO 
training  device. 

Methods 

The  analytic  methods  used  in  the  first 
phase  of  the  study  were  observation,  inter¬ 
view,  and  review  of  literature  and  documenta¬ 
tion. 

Carrier  landing  operations  were  observed 
both  ashore  and  at  sea.  More  than  250  Field 
Carrier  Landing  Practice  (FCLP)  approaches 
of  A-7  aircraft  were  observed,  and  over  700 
carrier  approaches  were  observed  from  the 
LSO  platform  aboard  the  USS  Coral  Sea  and 
the  USS  Enterprise. 

Informal  interviews  were  conducted  with 
approximately  20  LSOs.  Later,  structured 
interviews  of  approximately  two  hours  length 
were  conducted  with  LSOs  individually  and  in 
group  sessions.  The  purpose  of  the 
structured  interviews  was  to  specify  under 
what  circumstances  standard  NATOPS  LSO  calls 
would  be  used. 

Results 

The  analytic  data  were  used  to  develop 
the  LSO  decision  making  model  or  flow-chart 
model,  which  is  presented  fully  in  Borden 
and  McCauley  (1978).  Although  it  is  too 
lenghty  to  be  reproduced  fully  in  the 
present  paper,  an  excerpt  of  the  flow-chart 
model  is  given  in  Figures  1,  2,  and  3,  which 
show  the  Start  Loop,  Low  Procedure,  and 
Low  (A)  Procedure,  respectively.  The 
remaining  subroutines  in  the  model,  not 
included  in  this  paper,  are  the  following: 
High,  Line-Up,  Multiple  Deviation,  Sink, 
Climb,  Drift,  Speed,  Begin,  and  End. 

The  LSO  flow-chart  model  is  an  attempt 
to  represent  what  a  standardized  LSO  would 
do,  under  normal  circumstances,  in  response 
to  different  combinations  of  aircraft 
approach  dynamics.  A  distinct  lack  of 
standardization  in  the  LSO  cortmunity  assures 
that  no  single  LSO  would  behave  exactly  like 
this  model.  Although  it  is  a  model  based  on 
consensus,  every  individual  LSO  would 
disagree  with  something  in  the  model. 

An  elementary  version  of  the  model  also 
was  developed,  the  LSO  Call  Listing  Format 
Model.  This  format  lists  all  LSO  NATOPS 
calls  and  forms  a  matrix  of  62  calls  by  10 
aircraft  approach  variables.  For  each  LSO 
call,  the  values  of  aircraft  variables  that 
would  elicit  the  call  were  estimated.  The 
Call-Listing  Format  ignores  many  of  the 
sequential  dependencies  that  are  included  in 
the  flow-chart  model.  But  the  listing 
format  includes  important  situational  vari¬ 
ables  as  well  as  expected  pilot  response, 


related  LSO  strategies,  and  common  non¬ 
standard  (alternative)  LSO  calls  (Borden  and 
McCauley,  1978). 

Conclusions 

The  LSO  decision  making  model  is  far 
more  extensive  than  any  previous  effort  at 
LSO  modelling.  However,  the  estimated 
values  of  aircraft  approach  variables  that 
constitute  the  LSO  decision  criteria  were 
based  on  interview  data.  Empirical  measures 
of  the  aircraft  approach  and  the  coincident 
LSO  calls  were  needed  in  order  to  quantify 
the  LSO  decision  making  model.  The  LSO 
flow-chart  model  serves  as  a  form  of  task 
analysis  of  the  LSO's  job  during  a  final 
approach,  but  in  order  to  serve  as  a  criterion 
for  LSO-trainee  performance,  the  model 
requires  quantitative  support. 

STUDY  PHASE  II 

The  limitations  of  analytic  methods  were 
recognized  at  the  outset  of  the  study,  but 
obtaining  accurate  measures  of  approach 
performance  was  thought  to  be  cost  prohibi¬ 
tive,  requiring  instrumentation  of  both  the 
carrier  and  the  aircraft.  The  fact  that  no 
empirical  data  have  been  published  relating 
aircraft  approach  variables  to  LSO  perfor¬ 
mance,  attests  to  the  difficulty  of  obtaining 
quantitative  measures. 

During  the  analytic  evaluation,  HPR 
identified  a  measurement  method  called  the 
Photo  Data  Analysis  System  (PDAS)  that 
appeared  to  have  the  needed  capability  to 
accurately  measure  aircraft  approach  dynamics 
without  extensive  equipment  needed  at  the 
landing  area  or  aboard  the  aircraft.  The 
PDAS  is  installed  at  various  Naval  Test 
Facilities  to  analyze  the  initial  stage  of 
aircraft  weapon  performance,  called  "store 
separation. " 

The  PDAS  method  involves  cinematography 
of  the  moving  object,  sometimes  at  very  high 
frame  rates.  The  film  is  analyzed,  frame  by 
frame,  by  superimposing  the  f'lmed  image  of 
the  object  with  a  video  image  of  a  3-D  scale 
model  of  the  object.  The  two  images  are 
projected  onto  a  high  resolution  cathode  ray 
tube  by  video  mixing.  The  scale  model's 
position  is  controlled  by  the  PDAS  operation 
through  a  six-degree-of-freedom  model 
positioner.  When  the  images  are  superimposed, 
in  the  judgement  of  the  PDAS  operator,  data 
are  automatically  recorded  by  a  keypunch  and, 
later,  processed  by  computer.  Spatial 
measurements  in  six-degrees-of-freedom  are 
obtained  from  each  film  frame,  and  sequential 
analysis  of  frames  yields  measures  of  rates 
and  accelerations  (Cooper  and  Kingery,  1974). 


HPR  proposed  to  use  PDAS  for  obtaining 
measures  of  aircraft  approach  variables  while 
LSO  calls  were  being  recorded. 

Methods 

Several  data  collection  methods  for 
subsequent  PDAS  analysis  were  investigated, 
including  filming  approaches  aboard  a 
carrier,  analyzing  the  existing  Pilot/LSO 
Landing  Aid  Television  (PLAT)  videotapes  of 
carrier  landings,  and  filming  FCLP 
approaches.  The  first  two  methods  were 
eliminated  due  to  the  difficulty  of  platform 
motion  compensation.  PDAS  analyses  of 
ship-board  films  would  be  feasible  in  cases 
where  some  object  appeared  in  the  field  of 
view,  such  as  the  plane  guard  destroyer,  to 
provide  a  calibration  datum. 

The  present  study  limited  the  data 
collection  to  night  FCLP  approaches  of  A-7E 
aircraft  at  NAS  Lemoore.  FCLP  data  collec¬ 
tion  eliminated  the  motion  compensation 
problem,  and  enabled  multiple  LSOs  to 
simulate  "waving"  the  aircraft. 

Nearly  250  FCLP  approaches  were  filmed, 
and  one  to  four  qualified  LSOs  simulated 
controlling  each  approach  as  if  it  were  a 
carrier  approach.  The  film  and  the  audio 
recording  of  the  LSOs  were  synchronized  by 
a  time  code  generator. 

Analyses  and  Results 

Audio  Analysis.  HPR  analyzed  the  audio 
recordings  of  the  LSO  calls  during  FCLP  data 
collection.  Nearly  1000  LSO-waved  ap¬ 
proaches  (250  approaches  x  4  LSOs)  were 
analyzed.  Each  LSO  call  was  logged,  with 
the  time  of  onset  to  the  nearest  second. 

An  example  of  independent  action  from  3 
LSOs  on  the  same  bad  approach  is  given  in 
Table  1 . 


The  three  LSOs  all  detected  the 
(simulated)  dangerous  situation  and  used 
nearly  identical  calls  to  counteract  it. 

It  is  interesting  to  note,  however,  that  LSO 
*1  gave  two  calls  for  power  before  LSO  #3 
gave  the  first  one. 

Video  (PDAS)  Analysis 

At  Point  Muqu,  the  PDAS  analysis  of 
FCLP  films  has  been  delayed  by  a  ser  es  of 
events  includina  software  development, 
higher  priority  projects,  and  recalibration 
of  a  long  camera  carriage  track  following  an 
earthquake.  We  are  happy  to  report  that 
over  half  of  the  data  now  have  been  analyzed 
Presently,  the  data  are  limited  to  the  last 
15  seconds  of  each  approach.  PDAS  analysis 
of  the  data  for  the  first  7-10  seconds  of 
each  approach  is  scheduled  for  completion 
this  month. 

Figures  4  and  5  represent  approach 
attitude  by  range  for  two  approaches  called 
"good  flight"  and  "bad  fliqht,"  respectively 
The  "bad  flight"  (Figure  5)  is  the  same 
approach  as  in  Table  1,  where  the  calls  from 
3  LSOs  were  described.  The  data  rate  in  the 
PDAS  measures  is  two  per  second,  therefore, 
each  cross  on  all  of  these  figures  repre¬ 
sents  the  position/orientation  of  the 
aircraft  at  half-second  intervals.  Some 
calibration  has  yet  to  be  incorporated  in 
software,  so  these  and  subsequent  figures 
should  be  considered  only  as  prel iminary 
examples  of  PDAS  data  reduction. 

The  three  dimensional  position  of  the 
aircraft  ("good  flight")  as  a  function  of 
time  is  shown  in  Figure  6--ranoe,  line-up, 
and  altitude.  The  three  dimensional 
orientation  of  the  aircraft  over  time  is 
shown  in  Figure  7--pitch,  bank  angle,  and 
yaw.  The  software  calibration  for  yaw  has 
not  been  completed.  Similar  figures  are 


TABLE  1.  RECORDINGS  OF  3  LSOs  INDEPENDENTLY  "WAVING" 
AND  COMMENTING  ON  THE  SAME  BAD  FCLP  APPROACH 


TIME 

LSO  1 

LSO  2 

LSO  3 

20:41 : 

13 

A  little  Power 

41  : 

14 

Power. 

Power 

41  : 

15 

Wave-off 

Power 

Power 

41  : 

16 

Power!  Power! 

Power! 

41  : 

17 

Wave  it  off! 

LSO  Comments:  LSO  1  -  "May  have  rotated  and  hit  the  ramp  on  that  one;" 
LSO  2  -  "Looked  like  a  ramp  strike;" 

LSO  3  -  "Close  to  a  ramp  strike." 


ALTITUDE  (FT) 
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Figure  4.  Altitude  by  Range  for  "Good  Flight." 
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Figure  5.  Altitude  by  Range  for  "Bad  Flight." 
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available,  although  not  shown  here,  for 
velocities  over  time--sink  rate,  drift  rate, 
and  closure  rate  (ground  speed,  for  FCLP). 


W 


An  experimental  phase  plot  of  glide 
slope  by  rate-of-change  of  glide  slope  is 
shown  in  Figure  8  for  "good  flight."*  The 
same  type  of  phase  plot  is  shown  in  Figure  9 
for  "bad  flight."  Each  cross  represents 
the  combination  of  glide  slope  position 
error  and  rate-of-change  of  glide  slope 
position  (analogous  to  climb  or  sink). 
Consecutive  crosses  are  one-half  second 
apart,  and  the  terminal  (landing)  datum  is 
represented  by  a  circle.  Note  that  "good 
flight"  was  always  between  0  and  -12  feet 
from  a  3J°  glide  slope,  and  sink  rate 
(relative  to  glide  slope)  ranged  from  about 
21  ft.  per  second  sink  to  2i  ft.  per  second 
climb.  By  contrast,  "bad  flight"  (Figure 
9),  began  more  than  50  ft.  below  glide 
slope,  and  had  a  positive  rate  (climb 
relative  to  glide  slope)  throughout  the 
approach.  However,  the  climb  rate  decreased 
over  the  last  six  seconds  of  the  approach, 
and  was  insufficient  to  assure  ramp 
clearance.  At  approximately  2  seconds 
prior  to  the  circle,  representing  touch¬ 
down,  the  data  point  (aircraft  center  of 
gravity)  is  about  17  or  18  feet  below  glide 
slope. 

LSO  Model  1 ing 

The  aircraft  approach  measures  from 
PDAS,  when  combined  with  the  results  of  the 
LSO  calls  analysis,  will  provide  the  data 
necessary  to  estimate  LSO  call  criteria. 

Each  time  a  particular  LSO  call  is  used, 
the  aircraft  parameters  at  that  time  will 
be  saved  in  a  data  bank.  Using  these  data, 
statistical  distributions  will  be  obtained 
for  each  LSO  call,  enabling  the  mean  values, 
for  example,  of  the  critical  aircraft 
variables  to  be  specified  for  a  particular 
LSO  call.  Data  from  individual  LSOs  will 
be  compiled  separately,  and  this  will  allow 
individual  differences  to  be  noted  and 
analyzed. 

If  time  and  funding  allow,  techniques 
such  as  those  based  on  signal  detection 
theory  will  be  applied,  to  investigate 
differences  in  LSO  sensitivity  and  bias  to 
approach  deviations. 


♦Thanks  to  Don  Vreuls  of  Vreuls 
Research  Corporation  for  suggesting  the 
Phase  Plot. 


The  data  relating  aircraft  approach 
variables  to  LSO  actions  will  provide  a 
basis  for  establishing  LSO  traininq 
criteria  and  performance  measurement, 
either  for  an  automated  training  system, 
or  a  performance  evaluation  aid  for  the 
LSO  Instructor  in  a  non-automated  system. 
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LSO  PERFORMANCE  MEASUREMENT 
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RESEARCH  BACKGROUND 


The  Landing  Signal  Officer  (LSO)  as  an 
element  of  the  overall  aircraft  carrier  landing 
system,  contributes  substantially  to  the  quality 
of  landing  performance. 

This  is  especially  true  at  night  and  under 
less  than  optimum  conditions.  While  the  im¬ 
portance  of  the  LSO's  contribution  to  overall 
carrier  landing  system  effectiveness  is  not 
disputed,  relatively  little  is  known  about  the 
variability  of  his  performance  or  his  actual 
effect  on  the  system.  In  addition,  the  previ¬ 
ous  lack  of  objective  landing  performance  cri¬ 
teria  for  evaluating  carrier  landings  and  the 
contribution  of  the  LSO  to  the  quality  of  those 
landings  has  up  to  now  prevented  any  quanti¬ 
tative  estimates  of  LSO  performance  variability. 
Past  research  has  necessarily  focused  on  sub¬ 
jective  data  (e.g.,  LSO/pilot  opinion  question¬ 
naires)  and  intuition  in  the  absence  of  objective 
and  quantifiable  performance  criteria . 


and  dozens  of  LSOs  were  observed  and  mea¬ 
sured  on  aircraft  carriers  under  a  variety  of 
environmental  and  shipboard  conditions. 

Briefly ,  those  data  included  continuous  mea¬ 
surement  of  error  from  optimal  glideslope  and 
centerline  as  well  as  sink  speed,  airspeed, 
range,  deck  pitch,  arrestment  wire,  bolter  and 
waveoff  information  for  day  and  night  ap¬ 
proaches.  A  number  of  previous  studies  have 
reported  the  data  and  have  described  a  var¬ 
iety  of  pilot-related  carrier  landing  perfor¬ 
mance  topics  such  as  day  and  night  landing 
performance,  landing  performance  under  com¬ 
bat  conditions, 2  high  workload  operations, 2 
operational  measures  of  carrier  landing  system 
performance,1*  predictions  of  pilot  landing  per¬ 
formance  ^  and  visual  landing  aid  evalua¬ 
tion.®  Equally  important  to  this  research  were 


the  studie^of  criterion  development ,  test  and 
validation.  While  other  researchers  have 
attempted  qualitative  approaches  to  the  criter¬ 
ion  problem *0, 11  none  have  obtained  fleet  data 
in  objective  terms  related  to  job  performance. 


A  number  of  significant  training  and 
operational  problems  could  be  more  reliably 
approached  with  a  quantitative,  objective  mea¬ 
sure  of  LSO  performance.  For  example:  How 
much  training  is  necessary  for  satisfactory 
LSO  performance?  What  is  satisfactory  perfor¬ 
mance?  What  are  the  LSO  waveoff  boundaries 
for  different  aircraft?  What  is  the  transfer 
of  training  effectiveness  of  LSO  simulator 
training?  Does  LSO  performance  vary  across 
aircraft  type  and/or  model?  Is  LSO  perfor¬ 
mance  "better"  with  aircraft  they  themselves 
fly  or  is  ability  sufficiently  generalizable  such 
that  an  LSO  can  wave  all  aircraft  with  equal 
effectiveness?  Do  LSOs  lose  their  "seaman's 
eye"  with  extended  layoffs?  What  is  the  opti¬ 
mal  lay-off  interval  between  periods  of  LSO 
landing  experience?  How  much  "waving"  is 
necessary  to  bring  an  LSO  back  up  to  his 
normal  performance  level  after  extended  lay¬ 
offs?  Does  Performance  of  some  or  all  LSOs 
degrade  under  pitching  deck  conditions? 
Finally,  what  are  the  standards  for  LSO  per¬ 
formance?  Answers  to  these  and  other  equally 
critical  questions  could  be  provided  if  objec¬ 
tive  measures  of  LSO  performance  were  avail¬ 
able  . 


Past  LSO  studies1^  have  pinpointed 
through  questionnaire  data  that  perceptual 
ability  is  the  major  cause  of  attrition  during 
LSO  training.  A  related  study13  measured 
LSO  perceptual  ability  to  determine  the  range 
of  perceptual  ability  or  "seaman's  eye"  that 
exists  among  fleet  qualified  LSOs  as  well  as 
LSO  trainees.  More  recent  LSO  studies  have 
concentrated  on  analytic  techniques  to  enhance 
the  design  and  development  of  LSO  training 
systems.  One  study *•*  developed  an  LSO  deci¬ 
sion  model  for  application  in  an  automated  LSO 
training  device.  Another  report13  analyzed 
the  performance  capabilities  requirements  for 
the  same  automated  LSO  training  device.  The 
later  report  is  especially  useful  as  a  reference 
work  which  describes,  in  detail,  the  LSO  job 
and  current  training. 

The  research  reported  here  reports  speci¬ 
fically  on  fleet  qualified  LSOs  and  describes 
the  variability  of  their  performance  using  ob¬ 
jective  measures  of  carrier  landing  performance 
to  assess  and  describe  the  accuracy  of  their 
"seaman's  eye." 

TECHNICAL  APPROACH 


PREVIOUS  RESEARCH.  The  purpose  of  this 
research  was  to  investigate  methods  for  the 
objective  measurement  of  LSO  performance 
effects  in  carrier  landing  system  performance. 
In  previous  years  we  have  gathered  quantita¬ 
tive,  empirical  data  on  thousands  of  day  and 
night  carrier  approaches.  Experienced  and 
inexperienced  pilots,  seven  jet  aircraft  types. 


A  systematic,  analytic  approach  was  used 
in  developing  candidate  measures  of  LSO  per¬ 
formance.  The  approach  was  designed  to 
describe ,  in  quantitative  terms,  the  relative 
effectiveness  of  the  LSO  in  performing  his 
functions  in  carrier  recovery  operations.  By 
relating  the  performance  measures  to  LSO  air¬ 
craft  recovery  functions,  job  relevant  measures 


with  inherent  face  validity  can  be  provided. 

The  technical  objective  was  to  provide  practi¬ 
cal  performance  measures  that  could  be  used 
to  describe  relative  variations  in  LSO  perfor¬ 
mance  across  day  and  night  recovery  as  well 
as  providing  the  basis  for  the  eventual  devel¬ 
opment  of  LSO  job  performance  and  training 
standards. 

CARRIER  LANDING  SYSTEM  ANALYSIS.  The 
carrier  landing  system  is  defined  as  con sistin g 
of  four  primary  sub-systems--ship ,  aircraft, 
pilot,  LSO- -and  the  environment  in  which  it 
operates.  Each  contributes  to  total  system 
performance  effectiveness.  The  results  of  fac¬ 
toring  out  the  effects  of  each  of  the  sub¬ 
systems  and  their  relative  contributions  to 
total  system  performance  would  be  extremely 
useful,  but,  given  the  available  data  and  the 
complexity  of  system  interplay ,  does  not 
appear  feasible  at  present.  However,  the  per¬ 
formance  variability  within  each  of  the  sub¬ 
systems  can  and  has  been  explored. 

The  LSO  measures  considered  here  are  truly 
system  performance  measures — not  simply 
LSO  performance  measures — since  they  are  the 
combined  result  of  pilots  landing  airplanes  on 
ships  with  the  help  of  LSOs.  Although  the 
pilot  still  controls  the  aircraft,  and  probably 
is  the  prime  contributor  to  system  performance 
effectiveness,  the  LSO  is  considered  to  be  an 
active  control  element  in  the  system  which  pro¬ 
vides  feedback  to  the  pilot  on  the  quality  and 
acceptability  of  his  final  approach  performance. 

While  absolute  levels  of  pilot  or  LSO  per¬ 
formance  cannot  be  extracted  from  the  overall 
carrier  landing  system  performance,  relative 
differences  in  system  performance  as  a  func¬ 
tion  of  different  pilots  or  LSOs  can  be  accur¬ 
ately  described.  By  holding  the  ship  variable 
constant  (or  limited  to  one  ship  for  represen¬ 
tative  samples)  and  assuring  the  LSOs  have 
relatively  equal  exposure  to  samples  of  pilots, 
aircraft  types  and  models,  and  environment, 
then  any  quantitative  performance  differences 
observed  among  LSOs  can  be  considered  true 
differences  within  normal  statistical  sampling 
errors.  Similarly,  if,  on  a  single  ship,  a 
sample  of  pilots  flying  a  specific  aircraft  model 
are  equally  exposed  to  a  group  of  LSOs,  ob¬ 
served  differences  among  pilots  can  be  con¬ 
sidered  true  differences  within  sampling  error . 
Our  previous  research  has  isolated  the  effects 
of  pilot  performance,  aircraft  types  and  ship 
size.  This  study  will  focus  on  measurable  dif¬ 
ferences  in  LSO  performance  during  fleet  oper¬ 
ations  and  under  the  preceding  assumptions. 
Although  carrier  landing  performance  measures 
represent  total  system  performance,  LSO  re¬ 
lated  performance  differences  can  be  partialled 
out  of  the  systems  measures  if  all  LSOs  operate 
under  statistically  equivalent  ship,  aircraft 
and  pilot  conditions.  A  review  of  the  samples 
selected  from  the  carrier  landing  data  bank 
support  that  assumption . 


While  we  have  successfully  evaluated  ap¬ 
proach  measures  of  performance,  i.e.,  altitude 
and  lateral  error,  sink  rate,  speed,  etc.,  a 
simpler  performance  measure  was  desirable. 

One  such  measure  was  the  terminal  state  of  an 
approach ,  that  is  the  wire  number  of  arrested 
approaches,  bolters  or  waveoffs.  Approach 
terminal  state  measures  are  accurate  estimates 
of  approach  performance  due  to  the  correlation 
which  has  been  shown  to  exist  between  them. 

As  an  example,  distance  down  the  deck  (e.g., 
wire  number)  is  highly  correlated  ( .82)  with 
altitude  and  sink  rate  approach  performance 
measures  near  the  ramp  which  in  turn  are  high¬ 
ly  interrelated  with  one  another  at  distances  to 
1/2  mile  from  touchdown.®  Consequently, 
simple  wire  number,  by  virtue  of  its  correlation 
with  approach  performance ,  has  been  found  to 
be  a  very  reliable  estimator  of  approach  perfor¬ 
mance.  Thus,  one  way  to  estimate  LSO  effects 
on  approach  performance  is  through  measures 
of  terminal  landing  conditions  such  as  wire 
number,  bolters,  and  especially  LSO  initiated 
waveoffs.  The  latter  category  is  particularly 
relevant  to  the  LSO's  role  as  a  safety  monitor. 

Application  of  these  same  terminal  landing 
conditions  as  measures  of  pilot  landing  perfor¬ 
mance  has  been  successfully  tested  and  vali¬ 
dated9  and  forms  the  basis  for  the  technical 
approach  followed  here.  The  emphasis,  how¬ 
ever,  has  been  shifted  from  the  pilot  to  the 
LSO.  The  variability  of  LSO  performance  has 
been  partialled  out  of  the  carrier  landing  sys¬ 
tem  performance  to  allow  a  description  of  LSO 
performance  across  different  operating  ships, 
aircraft  and  environments. 

LSO  ROLE  IN  SYSTEM  PERFORMANCE.  The 
LSO  performs  two  distinct  functions  in  the  air¬ 
craft  carrier  landing  system.  According  to  the 
LSO  NATOPS  Manual  his  primary  responsibility 
is  the  safe  and  expeditious  recovery  of  aircraft 
aboard  ship. 

LSO  ROLE  AS  SAFETY  MONITOR.  The  LSO 
operates  in  parallel  with  other  system  elements 
as  a  safety  monitor  when  he  applies  various 
criteria  and  decides  to  wave  off  or  accept  an 
approach  during  the  final  seconds  before  touch¬ 
down  .  His  ultimate  criterion  is  successful  re¬ 
covery  .  The  LSO  as  a  safety  monitor  has  been 
investigated  to  some  extent  through  accident 
studies  in  terms  of  contributory  causal  factors 
but  not  objectively  in  terms  of  safety  criteria , 
and  how,  and  when  he  applies  them  and  with 
what  results. 

LSO  ROLE  IN  EXPEDITIOUS  RECOVERY.  The 
second  LSO  role,  one  which  has  been  neglected 
to  a  greater  extent  probably  due  to  the  inherent 
difficulties  in  its  investigation,  is  that  cr  expe¬ 
ditious  recovery.  Here  the  LSO  acts  as  an  ac¬ 
tive  control  element  operating  serially  with  other 
elements  in  the  landing  system.  As  an  active 
control  element,  the  LSO  continuously  monitors 
and  evaluates  aircraft  approach  performance 


and  initiates  voice  commands  to  the  pilot  as  he 
feels  necessary  to  expedite  safe  recovery . 

Those  commands  take  the  form  of:  1)  control 
commands,  e.g.,  power,  2)  directional  error, 
e.g.,  high/low,  3)  error  magnitude,  e.g., 
little  high/low,  and  4)  error  predictions,  e.g., 
going  high.  Functionally,  the  culls  are  cate¬ 
gorized  in  the  LSO  NATOPS  Manual  as  infor¬ 
mative,  precautionary,  and  imperative  calls. 

The  goal  of  ucting  as  a  controller  is  to 
assist  the  pilot  continuously  in  optimizing  air¬ 
craft  glideslope,  attitude,  speed,  sink  rate,  etc. 
such  to  achieve  arrestment  on  the  number  three 
target  wire.  To  the  extent  that  the  LSO  is  able 
to  "guide"  the  aircraft  into  what  might  be  called 
a  desired  tolerance  window  for  a  given  set  of 
circumstances,  landing  system  effectiveness  is 
enhanced.  Likewise,  to  the  extent  that  one 
LSO  is  better  able  to  monitor  and  guide  aircraft 
to  that  window  during  its  approach,  than  is 
another  LSO,  the  former  can  be  said  to  be  per¬ 
forming  "better"  as  a  system  element  than  the 
latter . 

METHOD 

DATA  SAMPLE  CHARACTERISTICS.  LSO  per- 
formance  data  analyzed  for  this  study  were  ob¬ 
tained  from  a  carrier  landing  data  bank  that 
totals  over  25,000  day  and  night  carrier  landing 
records  collected  under  a  wide  variety  of  ship, 
aircraft,  pilot,  LSO  and  environmental  condi¬ 
tions.  The  samples  selected  for  primary  analy¬ 
sis  were  taken  from  the  existing  data  bank ,  and 
therefore,  represent  an  ex  post  facto  analysis 
of  LSO  performance. 

The  samples  were  selected  for  analysis  on 
the  basis  of  several  criteria.  First  of  all,  a 
sample  was  needed  which  would  reflect  actual 
LSO  job  performance  under  operational  condi¬ 
tions  with  fully  qualified  pilots,  a  typical  variety 
of  jet  aircraft  and  a  suitable  number  of  day  and 
night  recoveries  across  a  normal  period  of  flight 
operations.  A  combat  sample  of  F4,  A6  and  A7 
aircraft  operating  on  CV-63  off  Yankee  station 
in  the  hottest  air  stages  of  the  Viet  Nam  war 
satisfied  the  above  criteria  and  represented  the 
final  culmination  of  LSO  training--experienced 
LSOs  waving  fleet  qualified  pilots  during  combat 
operations. 

A  second  sample  was  selected  to  explore 
LSO  training  performance  under  typical  Fleet 
Replacement  Squadron  (FRS)  conditions.  A 
combined  sample  consisting  of  FRS  LSOs  con¬ 
ducting  carrier  qualification  (CQ)  trials  for  A7 
and  F4  student  pilots  satisfied  the  need  for 
qualified  LSOs  working  with  inexperienced 
pilots.  Together,  these  two  samples  were  con¬ 
sidered  representative  of  the  wide  range  of  LSO 
job  performance  in  fleet  and  training  environ¬ 
ments. 

The  two  samples  are  characterized  aa 
follows: 


Sample  One:  Combat 


Sample  size 
approaches 
Aircraft  Types 
Environment  -- 


1718  (lay  /695  night 


•  Aircraft  Types  -  F4,  A6  and  A 7 

•  Environment  --  Combat  operations  in 

Viet  Nam 

•  Ship  --  CVA-G3.  3rd  line  period 

•  Pilots  --  Combat  ready 

•  LSOs  --  Experienced,  fleet  qualified 

Sample  Two:  Carrier  Qualification 

•  Sample  Size  --  150  night  landings 

•  Aircraft  Types  --  F4,  A7 

•  Environment  --  Carrier  Qualifications 

(CQ)  Training 

•  Ships  --  CVA-60,  CVAN-65 

•  Pilots  --  Inexperienced 

•  LSOs  —  Experienced 

•  Radar  records  with  synchronized  LSO 
calls 

LSO  PERFORMANCE  MEASURES.  In  order  to 
provide  objective  performance  data  for  the  two 
LSO  functions  a  number  of  candidate  perfor¬ 
mance  measures  used  in  previous  carrier  recov¬ 
ery  studies  were  analyzed.  These  measures 
were  categorized  as  either  terminal  or  final 
approach  measures. 

TERMINAL  LANDING  MEASURES.  Terminal 
landing  measures  used  to  describe,  summarize 
and  compare  LSO  performance,  mostly  for  the 
function  of  expeditious  recovery,  include: 

•  Boarding  rate  --  defined  as  the  number 
of  wire  arrestments  divided  by  the  total 
number  of  approaches  times  100. 

•  Unsuccessful  approach  rate  —  defined 
as  the  number  of  technique  waveoffs 
and  the  bolters  divided  by  the  total 
number  of  approaches  times  100.  This 
measure  may  also  be  calculated  by  sub¬ 
tracting  the  boarding  rate  from  100  per¬ 
cent. 

•  Final  approach  outcome  distribution  — 
a  descriptive  technique  used  to  display 
the  relative  percentages  of  1,  2,  3  and 
4  wire  arrestments,  technique  waveoffs 
and  bolters. 

•  Landing  Performance  Score  (LPS)  —  a 
weighted  measure  derived  from  a  pre¬ 
vious  study®  which  reflects  the  relative 
importance  of  each  type  of  final  approach 
outcome.  The  weightings  used  for  the 
LPS  are  based  on  an  interval  scale  and 
represent  a  single  quantitative  value 

for  each  landing  category. 

•  Final  approach  outcome  profile  —  a  com¬ 
prehensive  summary  of  all  terminal  land¬ 
ing  measures  by  LSO. 

FINAL  APPROACH  RADAR  MEASURES.  All  of 
the  above  measures  and  descriptive  techniques 
provide  a  means  to  compare  the  terminal  outcome 
performance  of  a  series  of  approaches.  Although 
these  techniques  summarize  an  LSO's  perfor¬ 
mance  as  a  safety  and  control  monitor  they  do 


not  describe  the  final  approach  leading  up  to 
the  landing  outcome.  By  obtaining  knowledge 
of  the  final  approach  process  one  may  obtain  a 
better  understanding  of  the  LSO's  function 
during  carrier  recovery.  One  technique  for 
obtaining  final  approach  information  is  through 
the  collection  of  precision  radar  records  (SPN- 
42)  along  with  synchronized  LSO  calls.  These 
latter  data  readily  provide  knowledge  of  an 
LSO's  effectiveness  as  a  safety  monitor  and 
control  monitor  and  are  described  in  the  sample 
of  CQ  data . 

LSO  VOICE  CALL  ANALYSIS.  LSO  voice  calls 
were  manually  recorded  for  a  small  sample  of 
CQ  trials  during  which  each  LSO  verbal 
response  was  annotated  to  the  radar  record  to 
provide  a  match-up  of  aircraft  flight  path  and 
LSO  call  response.  The  records  are  analyzed 
for  information  on  the  type,  frequency,  range, 
and  effectiveness  of  different  calls  by  LSO  and 
aircraft  type. 

ANALYSIS  OF  UNSUCCESSFUL  APPROACHES. 
Unsuccessful  approaches  such  as  technique 
waveoffs  and  bolters  are  analyzed  for  informa¬ 
tion  on  the  effectiveness  of  LSOs  as  safety 
monitors.  The  range  at  which  the  waveoff  was 
initiated  by  the  LSO,  the  percentage  of  unsuc¬ 
cessful  approaches  among  LSOs  and  an  analysis 
of  LSO  control  effectiveness  for  unsuccessful 
approaches  are  used  for  descriptive  and  com¬ 
parative  purposes. 

DATA  LIMITATIONS.  As  in  many  field  studies, 
experimental  manipulation  of  independent  var¬ 
iables  of  interest  was  not  under  our  control. 
Consequently,  the  vast  amount  of  data  in  the 
data  bank  were  partitioned  after  the  fact  into 
categories  appropriate  for  the  specific  LSO 
analyses  described  below.  One  of  the  principal 
classifications  of  data  was  that  of  identifying 
LSOs  waving  each  approach  for  two  major  data 
samples. 

The  reliability  of  descriptive  statistical 
measures  may  be  a  function  of  the  sample  size. 
While  they  are  typically  large  for  gross  cate- 
fories,  e.g.,  day  vs.  night,  sample  size  reduces 
rapidly  with  further  partitioning,  e.g.,  day  vs. 
night  x  LSO  x  aircraft  type.  In  some  cases 
partitioning  resulted  in  sub-sample  sizes  too 
small  for  meaningful  comparisons.  Consequently, 
some  comparisons  of  subcategories  could  not  be 
reasonably  made  because  of  the  small  number  of 
cases  involved.  Where  possible,  which  was  for 
the  majority  of  cases,  LSO  x  aircraft  was  com¬ 
pared,  and  no  systematic  differences  were  noted 
other  than  those  mentioned  in  the  analysis  sec¬ 
tion  . 

In  making  reliable  comparisons  among  LSOs, 
it  was  necessary  to  assume  that  LSOs,  in  the 
long  run,  wave  different  aircraft  types  with 
equal  frequency  since  some  aircraft  are  known 
to  be  better,  on  the  basis  of  accident  rates  and 
boarding  rates,  than  others.  The  assumption 


was  tested  by  comparing  the  frequency  distri¬ 
bution  of  aircraft  types  across  LSOs  and  ap¬ 
pears  valid  for  this  data  sample.  No  LSO  waved 
one  aircraft  type  significantly  more  frequently 
than  another. 

It  was  also  necessary  to  assume  that  LSOs 
waved  pilots  of  equivalent  ability.  Fortunately, 
this  assumption  does  lend  itself  to  direct  veri¬ 
fication  since  identification  of  "good"  and  "poor” 
pilots  was  possible  through  previous  data  anal¬ 
ysis.  The  assumption  appears  logically  valid, 
i.e.,  there  is  no  reason  to  suspect  that  LSOs 
systematically  wave  only  good  or  poor  pilots. 
Rather,  each  pilot  was  considered  qualified  to 
land  aboard  ship  in  that  aircraft  type.  Thus, 
it  was  assumed  and  verified  that  pilot  ability 
did  not  systematically  bias  LSO  comparisons. 

RESULTS 

COMBAT  SAMPLE .  Results  from  the  analysis 
of  aircraft  recovery  during  combat  operations 
off  the  coast  of  Viet  Nam,  where  both  pilots 
and  LSOs  were  highly  trained  and  experienced, 
are  shown  in  a  series  of  figures  which  describe 
LSO  performance  variations  during  night  car¬ 
rier  recovery.  Data  are  presented  in  four 
figures  to  provide  a  comprehensive  summary  of 
LSO  performance: 

•  Boarding  Rate 

•  Unsuccessful  Approaches 

•  Landing  Performance  Scores  (LPS) 

•  Final  Approach  Outcome  Profiles 

NIGHT  BOARDING  RATE.  LSO  night  boarding 
rates  are  illustrated  in  Figure  1  across  ten  LSOs 
with  a  sufficient  number  of  night  approaches 
(>25)  to  justify  calculation  of  descriptive  means. 
For  all  LSOs  the  boarding  rate  was  93  percent, 
ranging  from  a  high  of  100  percent  successful 
(LSO  #1)  to  a  low  boarding  rate  of  88  percent 
(LSO  #6).  The  variability  range  was  only  12 
percent  between  highest  and  lowest  performing 
LSOs  for  this  line  period. 


LSO  * 

Figure  1.  Night  Boarding  Rate  by  LSO  Across 
All  Aircraft  Types  for  Combat  Sample. 
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In  terms  of  LSO  functions,  since  there 
were  no  landing  accidents,  each  LSO  was  suc¬ 
cessful  in  his  safety  monitor  role.  Looking  at 
his  role  in  expeditious  recovery  a  93  percent 
night  boarding  rate  was  achieved.  This  can  be 
considered  a  very  successful  line  period  com¬ 
pared  with  fleet-wide  night  boarding  rates  that 
averaged  in  the  low  eighties  for  most  operations. 

NIGHT  UNSUCCESSFUL  APPROACHES.  An¬ 
other  way  to  look  at  LSO  performance  besides 
boarding  rate  is  by  analyzing  unsuccessful 
approaches.  In  this  case  unsuccessful  ap¬ 
proaches  are  defined  as  technique  waveoffs  or 
bolters. 

LSO  unsuccessful  approaches  are  graphed 
in  Figure  2.  Across  LSOs  7  percent  for  all 
approaches  were  unsuccessful,  ranging  from  5 
percent  for  bolters  to  2  percent  for  waveoffs. 
LSO  number  one  had  no  unsuccessful  ap¬ 
proaches  while  LSO  number  six  had  12  percent 
of  his  approaches  result  in  either  bolters  (10 
percent)  or  waveoffs  (2  percent).  LSO  number 
two  had  no  bolters  while  LSO  number  five  had 
no  waveoffs. 
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Figure  2.  Night  Unsuccessful  Approaches  by 
LSOs  Across  All  Aircraft  Types  for  Combat 
Sample . 

NIGHT  LANDING  PERFORMANCE  SCORES  (LPS). 
Landing  Performance  Scores  present  the  first 
opportunity  to  assign  a  composite  weight  to  the 
quality  of  overall  LSO  performance.  The  LPS 
is  a  weighted  concensus  of  the  'goodness'  of 
terminal  outcome  based  on  the  number  3  target 
wire.  All  other  landing  outcomes  have  scores 
relative  to  that  of  the  target  wire. 

LSO  LPS  scores  are  presented  in  Figure  3. 
Overall.  LSOs  averaged  4.95  for  LPS  scores 


across  all  aircraft.  LSOs  varied  in  their  LPS 
scores  from  5.31  for  LSO  number  one,  to  4.54 
for  LSO  number  ten.  Most  LSOs  deviated  only 
slightly  from  the  mean  with  the  exception  of 
the  high  and  low  LPS  scores  of  LSOs  number 
one  and  number  ten,  respectively. 


Figure  3.  Mean  LPS  by  LSO  at  Night  Across 
All  Aircraft  Types  for  Combat  Sample. 

FINAL  APPROACH  OUTCOME  PROFILES.  LSO 
final  approach  outcome  profiles  present  in  a 
composite  form  all  the  data  previously  presented 
in  separate  figures.  Each  column  of  data  pre¬ 
sents  the  boarding  rate,  LPS,  final  outcome  dis¬ 
tribution  and  a  relative  graphic  portrayal  of  all 
outcomes  across  all  LSOs  (see  Figure  4).  The 
profiles  represent  a  comprehensive  method  of 
formatting  all  LSO  performance  measures  for 
quick  reference,  summary,  comparison  and  con¬ 
trast.  This  is  especially  useful  in  identifying 
training  strengths  (number  3  wire)  and  potential 
weaknesses  (waveoffs.  bolters,  number  1  wires). 

For  example,  the  boarding  rate,  LPS  and 
wire  arrestment  frequency  data  can  be  obtained 
quickly  and  easily  by  locating  the  LSO  number 
and  then  checking  that  particular  column.  The 
average  performance  for  all  LSO's  shows  that 
the  largest  percentage  of  outcomes  was  on  the 
number  three  target  wire  (40  percent)  and  the 
fewest  was  on  technique  waveoffs  (2  percent). 
The  distribution  reflects  a  high  number  of  tar¬ 
get  wire  arrestments,  a  small  percentage  of 
number  one  wire  arrestments  (short  landings) 
and  a  combined  84  percent  of  all  approaches 
terminating  in  number  2.  3.  or  4  wires.  Wave¬ 
offs,  bolters  and  number  one  wire  arrestments 
are  minimized  (16  percent).  Each  individual 
LSO  can  also  be  compared  to  the  sample  mean  as 
well  as  with  every  other  LSO.  Using  LSO 
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Figure  4.  Night  Final  Approach  Outcome  Profile  by 
LSO  Across  All  Aircraft  Types  for  Combat  Sample. 
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number  one  and  number  ten  as  examples,  their 
performance  differences  are  readily  apparent 
and  the  relative  differences  between  and  among 
l.SOs  are  easily  identified. 

LSO  VOICE  CAM.  ANALYSIS.  Part  of  the  data 
collected  during  tiio  night  CQ  sample  consisted 
of  precision  radar  records  of  aircraft  final 
approach  parameters  and  synchronized  with 
t.SO  voice  calls  i’e\  orded  during  the  actual  final 
approach.  These  data  present  the  first  oppor¬ 
tunity  to  analyze  LSO  voice  calls  made  to  pilots 
during  final  approach.  The  voice  calls  are  for 
FI  and  A7  night  CQ  operations.  A7  CQ  data 
were  collected  during  pitching  deck  conditions, 
while  F4  data  were  recorded  for  normal,  high 
visibility,  CQ  recoveries. 

LSO  voice  calls  are  analyzed  by  the  follow¬ 
ing  methods  which  are  intended  to  provide  both 
descriptive  and  diagnostic  information  on  the 
nature  of  LSO  calls,  their  variability  across 
l.SOs  and  their  effectiveness  as  evaluated  by 
various  criteria  . 

•  Frequency  of  LSO  voice  calls 

•  Distribution  of  calls  by  aircraft  type 

•  Boarding  rate  with  and  without  LSO 
calls 

•  LSO  calls  by  range  interval 

•  Type  of  LSO  voice  calls 

•  Ambiguous  LSO  voice  calls 

•  LSO  calls  by  NATOPS  taxonomy 

FREQUENCY  OF  LSO  VOICE  CALLS.  The  fre¬ 
quency  of  LSO  voice  calls  made  during  each 
night  CQ  approach  for  A 7  and  F4  aircraft  were 
analyzed.  Seventy-one  percent  of  all  night  ap¬ 
proaches  had  one  (37%)  or  two  (30%)  LSO  voice 
calls,  14  percent  had  three  calls,  10  percent  had 
four  or  more  calls,  and  five  percent  had  no  calls 
at  all . 

DISTRIBUTION  OF  CALLS.  The  distribution  of 
LSO  calls  by  aircraft  type  was  found  to  differ 
for  F4  and  A 7  CQ  approaches.  The  majority  of 
A  7  approaches  (55  %)  had  only  one  call  while 
two  calls  was  the  largest  category  for  F4  ap¬ 
proaches  (46%).  Only  three  percent  of  all  A7 
approaches  and  20  percent  of  F4  approaches 
had  four  or  more  LSO  calls  per  approach .  This 
method  of  portraying  LSO  voice  calls  by  fre¬ 
quency  of  calls  and  percentage  of  approaches 
with  one  or  more  cadis  is  a  useful  way  to  break¬ 
down  the  LSO  call  data  into  more  meaningful 
training  feedback  information  on  how  frequency 
of  calls  varies  by  aircraft  and  LSO  combinations. 

BOARDING  RATE.  Probably  a  more  operation¬ 
ally  useful  method  to  analyze  LSO  voice  calls  is 
to  see  the  effect  the  number  of  calls  has  on 
boarding  rate  and  LPS  by  aircraft  type.  With 
one  or  more  calls,  boarding  rate  for  A 7  and  F4 
CQ  approaches  was  74  percent  and  seems  to  in¬ 
dicate  that  some  LSO  feedback  enhances  suc- 
cessful’recovery .  When  no  LSO  voice  calls  were 
made  boarding  rate  was  only  55  percent. 


When  looked  at  separately,  F4  LSOs  aver¬ 
aged  2.3  calls  per  approach  for  a  boarding  rate 
of  83  percent  and  an  LPS  of  4.2.  A7  LSOs 
averaged  1.5  calls  per  approach  for  boarding 
rate  of  55  percent  and  an  LPS  of  3.3.  Recall 
that  A7  recoveries  were  made  during  CQ  under 
pitching  decks.  It  seems  that  fewer  LSO  calls 
per  approach  were  made  for  A 7s  operating 
under  pitching  decks  than  for  F4s  operating 
under  normal  recovery  conditions.  Part  of  the 
explanation  may  lie  in  the  philosophy  of  some 
LSO  CQ  training.  In  order  to  develop  an  auto¬ 
nomous  pilot,  at  times  LSOs  withhold  voice  calls 
to  encourage  pilot  self  reliance  and  avoid  over 
dependence  on  the  LSO  for  approach  informa¬ 
tion.  This  is  especially  true  under  CQ  training 
conditions  when  the  pilot  is  first  learning  the 
skills  and  judgement  to  land  aboard  ship.  While 
this  training  philosophy  may  not  be  consistent 
across  LSOs,  it  may  be  one  reason  why  A 7  LSOs 
had  fewer  calls  per  approach  than  did  F4  LSOs 
even  though  more  severe  environmental  condi¬ 
tions  prevailed  for  the  A7  CQ  trials.  In  any 
case,  one  point  to  remember  is  that  the  methods 
used  to  analyze  the  LSO  voice  calls  can  be  used 
to  detect  variations  in  LSO  training  philosophy 
and  appear  to  be  useful  in  describing  and 
understanding  LSO  voice  call  behaviors. 

RANGE  INTERVAL.  The  range  interval  of  LSO 
calls  were  delineated  into  final  approach  phases 
— at  the  start,  in  the  middle,  in  close  and  at 
the  ramp.  A  comparison  found  that  A 7  LSOs 
provided  just  over  20  percent  of  their  voice 
calls  in  the  vicinity  of  the  ramp  as  contrasted 
to  under  10  percent  for  the  F4.  Other  differ¬ 
ences  were  only  dight  with  a  gradual  increase 
in  LSO  calls  as  the  aircraft  nears  touchdown. 
Range  interval  plots  may  be  useful  in  depicting 
variation  in  aircraft/LSO  interactions  and  pro¬ 
viding  data  to  allow  determination  of  a  standard 
range  based  response  set  for  different  aircraft 
types.  In  these  data  it  appeared  that  the  A 7 
LSOs  provided  equal  information  all  the  way  to 
the  ramp,  while  F4  LSOs  tended  to  provide  more 
information  early  in  the  approach.  These  dif¬ 
ferences  are  probably  due  to  different  aircraft 
response  times.  Again,  the  A7  had  pitching 
decks  which  may  also  have  contributed  to  a 
higher  percent  of  calls  made  at  the  ramp. 

TYPE  OF  LSO  CALLS.  The  types  of  LSO  calls 
were  further  categorized  by:  glideslope  calls, 
power  calls,  attitude  calls,  ambiguous  calls  and 
other  calls.  LSO  F4  voice  calls  were  dominated 
by  power  (28  percent)  and  glideslope  calls  (28 
percent).  A7  calls  were  mostly  glideslope  calls 
(51  percent)  with  only  14  percent  power  calls. 
This  method  of  analyzing  voice  calls  seems  to 
indicate  its  potential  for  describing  variations 
of  LSO  calls  with  aircraft  type  due  to  response 
characteristics.  In  the  example  used  there  are 
clear  differences  in  the  frequency  of  call  types 
between  F4  and  A 7  aircraft  types. 

Another  interesting  category  of  calls  are 
indicated  by  what  we  term  "ambiguous"  calls. 


They  represented  16  percent  of  all  F4  calls  and 
24  percent  of  all  A 7  LSO  voice  calls.  An 
example  of  an  ambiguous  call  is  "Fly  the  ball." 
Ambiguous  calls  require  the  pilot  to  decide 
where  he  is  on  the  final  approach  before  he 
responds  to  the  call.  The  most  frequently  used 
ambiguous  call  was  "Fly  the  ball"  which  made  up 
44  percent  of  all  these  types  of  culls. 

A  brief  analysis  of  ambiguous  "Fly  the  ball" 
calls  revealed  that  half  the  calls  were  made 
when  the  aircraft  was  above  glideslope  and  half 
when  the  aircraft  was  below  glideslope.  When 
compared  to  directional  calls  such  as  "Don't  go 
high."  ambiguous  calls  showed  a  lower  boarding 
rate  (67%)  and  LPS  (3.8)  compared  with  a  high¬ 
er  boarding  rate  (75%)  and  LPS  (4.2)  for 
"Don’t  go  high"  calls.  The  tentative  conclusion 
reached  is  that  specific  types  of  LSO  voice  calls 
may  influence  terminal  performance,  especially 
the  use  of  ambiguous  calls.  A  sizeable  percent¬ 
age  of  ambiguous  calls  were  made  during  CQ 
training  and  the  methods  used  to  describe  the 
data  allows  computation  of  their  frequency  and 
effect  on  terminal  performance. 

LSO  CALLS  BY  NATOPS  TAXONOMY.  The  LSO 
NATOPS  Manual  has  four  categories  of  voice 
calls:  informative,  precautionary,  imperative 
and  non-standard. 

The  largest  percentage  of  calls  for  the  F4s 
were  imperative  calls  (54%)  and  for  the  A 7, 
non-standard  calls  (58%).  Non-standard  calls 
are  those  that  do  not  conform  to  the  official 
NATOPS  taxonomy  and  are  somewhat  like  ambig¬ 
uous  calls,  which  would  be  included  as  non¬ 
standard  calls  in  most  cases. 

Measurement  and  analysis  techniques  such 
as  those  presented  in  this  analysis  of  LSO 
voice  calls  should  prove  useful  in  formulating 
various  LSO  training  standards  for  voice  trans¬ 
missions  monitoring  and  assessing  their  applica¬ 
tion  in  fleet  and  training  operations. 

LSO  CONTROL  EFFECTIVENESS 

LSO  control  effectiveness  was  analyzed 
using  night  final  approach  data  recorded  by 
twin  precision  radars  augmented  with  time  syn¬ 
chronized  LSO  voice  calls  to  the  pilot.  Three 
examples  of  LSO  control  effectiveness  were  pre¬ 
pared.  One  example  was  a  vertical  and  lateral 
error  plot  and  LSO  voice  calls  for  what  is  con¬ 
sidered  to  be  an  effective  LSO/pilot  interaction. 
LSO  control  effectiveness  means  that  the  LSO 
calls  were  informative,  timely  and  useful  in 
assisting  the  pilot  to  a  successful  number  three 
target  wire  recovery.  Other  examples  were 
effective  LSO  calls,  poor  pilot  response  and 
late  LSO  calls  coupled  with  good  pilot  response. 
The  examples  were  analyzed  in  detail  to  provide 
an  indication  of  how  useful  radar  records  with 
time  synchronized  LSO  voice  calls  are  in  analy¬ 
zing  LSO  control  effectiveness.  The  synchron¬ 


ized  voice  and  error  traces  permit  us  to  infer 
the  LSOs  perceptual  ability  and,  therefore,  to 
describe  his  "seaman's  eye."  The  implications 
of  this  technique  for  LSO  training,  especially 
simulator  training  now  under  development,  are 
enormous  in  terms  of  assessing  LSO  precision 
and  accuracy  in  night  carrier  recovery  opera¬ 
tions  and  providing  diagnostic  training  feedback 
for  performance  enhancement. 

ANALYSIS  OF  LSO  TECHNIQUE  WAVEOFFS. 

One  useful  way  of  describing  LSO  effectiveness 
in  the  area  of  safety  monitor  is  to  analyze  tech¬ 
nique  waveoffs.  For  the  night  CQ  sample  of 
150  F4  and  A7  recoveries,  there  were  fifteen 
LSO  technique  waveoffs.  Ten  percent  (15)  of 
the  night  CQ  sample  consisted  of  LSO  technique 
waveoffs.  Twelve  of  the  15  were  A7  (pitching 
deck)  waveoffs.  Sixty  percent  of  the  waveoffs 
were  between  300  and  500  feet  from  touchdown. 
Seventy -five  percent  were  within  600  feet.  At 
the  other  extreme,  25  percent  were  beyond  750 
feet  from  touchdown.  When  the  altitude  error 
of  each  waveoff  was  calculated,  eight  were 
found  to  be  above  glideslope  with  a  mean  of  13 
feet  high  and  a  range  of  496  feet  from  touch¬ 
down.  Seven  were  below  glideslope  with  a  mean 
of  9  feet  low  and  a  waveoff  range  of  668  feet 
from  touchdown.  From  this  analysis  it  appears 
that  low  waveoff  passes  are  terminated  sooner 
than  high  waveoff  approaches.  Also,  the  aver¬ 
age  deviation  from  glideslope  was  less  for  low 
approaches  which  are  waved  off  than  it  was  for 
high  approach  waveoffs.  Thus,  it  appears  the 
LSO  acting  as  a  safety  monitor  may  have  differ¬ 
ent  criteria  for  high  and  low  waveoffs.  The 
promise  of  empirically  identifying  the  waveoff 
cluster  ranges  would  allow  a  more  objective 
determination  of  LSO  waveoff  boundaries  by 
aircraft  type.  Eventually,  objective  waveoff 
standards  could  be  developed  using  this  mea¬ 
surement  technique. 

Another  way  to  describe  LSOs  effectiveness 
in  monitoring  final  approaches  is  by  looking  at 
what  we  describe  as  satisfactory  landings  (#2, 
#3,  and  #4  wire  arrestments),  oi  conversely, 
unsatisfactory  landings  (bolters,  waveoffs,  and 
#1  wire  arrestments).  In  the  absence  of  acci¬ 
dent  data,  this  method  could  provide  some  feed¬ 
back  on  the  central  tendency  or  accuracy  of 
LSO  performance  in  arresting  aircraft  on  the 
number  three  target  wire.  For  the  combat 
sample  84  percent  of  the  night  approaches  were 
so-called  satisfactory  landings,  and  only  16  per¬ 
cent  unsatisfactory. 

DISCUSSION 

LSO  performance  was  analyzed  within  a 
carrier  landing  system  framework.  Performance 
measures  related  to  the  two  primary  LSO  func¬ 
tions —  safe  and  expeditious  recovery  of  aircraft 
--were  developed  and  applied  to  two  samples  of 
recovery  data.  As  a  safety  monitor,  the  LSO 
must  detect  unsafe  aircraft  deviations  from  the 


LSO  carrier  landing  performance  can  be 
assessed  using  these  preliminary  standards  or 
similar  standards  agreed  upon  as  appropriate 
by  command  or  training  personnel.  While  these 
data  do  not  represent  a  consensus  as  to  the 
desirable  levels  of  LSO  performance,  data  col¬ 
lected  across  these  candidate  criteria  could 
provide  the  information  necessary  to  develop 
LSO  performance  standards  for  combat  as  well 
as  other  operational  or  training  conditions. 


optimum  glideslope  and  waveoff  all  approaches 
that  present  an  imminent  danger  to  the  safe 
recovery  of  aircraft  and  crew.  As  a  control 
monitor,  the  LSO  is  expected  to  expedite  re¬ 
covery  by  continuously  assisting  the  pilot  in 
the  pursuit  of  a  successful,  and  timely,  ap¬ 
proach.  While  these  functions  are  not  exclu¬ 
sive  of  each  other  we  have  used  different 
performance  measures  to  describe  the  relative 
variability  f  LSO  performance  within  each 
functional  role . 

The  most  promising  feature  of  the  methods 
presented  is  their  empirical  base.  While  the 
measures  have  been  derived  from  information 
extracted  from  an  existing  carrier  landing  data 
bank  their  application  to  LSO  performance  mea- 
surement  was  successful  in  all  respects.  The 
use  of  operational  measures  such  as  boarding 
rate,  approach  distributions  and  unsuccessful 
approaches,  as  well  as  research  measures 
represented  by  the  LPS,  provide  a  comprehen¬ 
sive  summary  of  LSO  performance  in  his  two 
functional  modes. 

The  emphasis  in  this  report,  however,  is 
placed  on  the  performance  measurement  tech¬ 
niques  rather  than  the  LSO  performance  data 
itself.  While  samples  of  LSO  performance  dur¬ 
ing  combat  and  CQ  training  were  chosen  to 
represent  the  broad  range  of  LSO  activity  in 
carrier  recovery  operations,  the  primary  inten¬ 
tion  was  to  apply  and  test  the  measurement 
techniques  and  not  to  assess  LSO  performance 
per  se.  During  the  course  of  reviewing  the 
data  with  staff,  fleet  and  training  LSOs  it  was 
apparent  that  some  of  the  performance  descrip¬ 
tions  were  pertinent  to  current  LSO  training 
especially  analysis  of  LSO  voice  calls  and  un¬ 
successful  approaches.  Several  suggestions 
were  also  made  as  to  how  the  data  might  be 
used  to  develop  performance  standards  for 
various  stages  of  LSO  training.  For  example, 
LSO  performance  in  the  combat  sample  could  be 
used  as  preliminary  operational  standards.  The 
high  performance  levels  attained  by  the  airwing 
LSOs  represents  a  standard  of  excellence,  or 
goal  for  combat  deployment.  Consider  the 
following  performance  criterion  levels  fo"  that 
data  across  the  two  LSO  functions. 

•  Ls0  Expedite  Recovery  Function 

Day  Boarding  Rate:  97% 

Night  Boarding  Rate:  93% 

LPS:  5.03  Day,  4.95  Night 
Bolters:  2%  Day  .  5%  Night 
%  Target  Wires:  41%  Day,  41%  Night 

•  LSO  Safety  Monitor  Function 
Accidents:  Zero 

Technique  Waveoffs:  l%Day,  5%  Night 
Satisfactory  Landings:  87%  Day  , 

84%  Night 


The  analysis  of  LSO  performance  during 
CQ  trials  using  radar  records  with  time  synch¬ 
ronized  LSO  voice  calls  is  a  most  promising 
technique  for  measuring  LSO  control  effective¬ 
ness  and  for  training  and  performance  enhance¬ 
ment.  With  beginning  LSOs  the  technique 
appears  useful  for  describing  actual  final 
approaches  with  annotated  LSO  calls.  These 
records  could  be  used  to  familiarize  the  student 
LSO  with  correct  use  of  voice  calls,  the  nature 
of  pilot  responses,  and  appropriate  final 
approach  corrective  actions.  Various  aircraft 
approaches  and  LSO  calls  could  be  collected  in 
the  fleet  for  a  wide  variety  of  aircraft,  weather 
and  environmental  conditions.  They  would  pro¬ 
vide  trainee  LSOs  with  operational  examples  of 
experienced  LSOs  waving  aircraft  under  vari¬ 
able  conditions  found  in  the  fleet.  The  measure¬ 
ment  technique  would  also  seem  invaluable  for 
any  LSO  simulator  training.  Additional  techni¬ 
ques  to  measure  LSO  safety  monitor  functions 
include  the  analysis  of  technique  waveoffs 
which  could  prove  useful  in  empirically  describ¬ 
ing  waveoff  range  data  and  LSO  waveoff  bound 
aries  if  based  on  a  large  enough  sample  of 
operational  landings. 

The  analysis  of  LSO  voice  calls  is  a  fruit 
ful  method  to  collect  data  for  standards  develop¬ 
ment.  The  preliminary  description  of  LSO  voice 
calls  presented  here  has  raised  a  few  training 
questions.  What  is  the  LSO  community  concen¬ 
sus  on  different  LSO  use  of  non  standard  or 
ambiguous  calls?  Also,  what  types  of  calls  are 
most  appropriate  for  different  aircraft  approach 
conditions?  Continued  collection  and  update  of 
LSO  performance  data  would  seem  mandatory  to 
provide  answers  to  a  host  of  LSO  voice  call 
effectiveness  questions.  The  measurement  tech¬ 
nique  is  available  and  the  operational  and  train¬ 
ing  communities  would  appear  to  benefit  from 
its  continued  development  and  application. 

Finally,  some  thought  should  be  given  to 
the  development  and  implementation  of  a  carrier 
landing  performance  measurement  system  aboard 
ship.  A  manual  version  of  such  a  system  now 
exists  and  was  used  to  collect  the  data  presented 
here.  Such  a  system  would  provide  objective 
measures  of  both  pilot  and  LSO  performance  for 
day  and  night  carrier  recovery.  With  a  ship¬ 
board  performance  measurement  system,  preci¬ 
sion  radar  data,  synchronized  with  LSO  voice 
calls,  could  be  used  for  a  detailed  assessment  of 
LSO  performance  effectiveness  in  terms  of  the 
prediction  and  control  of  aircraft  recovery. 
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INTRODUCTION 

The  Navy  has  recently  procured  an  LSO 
training  station  for  the  A7E  Night  Carrier 
Landing  Trainer  (NCLT).  This  station  is 
called  the  LSO  Reverse  Display  (LSORD).  It 
provides  a  simulated  night  carrier  approach 
scene,  as  viewed  from  the  LSO  platform,  and 
provisions  for  "waving"  approaches  flown  by 
a  pilot  in  the  NCLT.  An  NTEC-sponsored 
project  is  underway  to  evaluate  the  training 
effectiveness  of  the  LSO  Reverse  Display. 
This  paper  presents  preliminary  findings 
which  have  resulted  from  early  evaluation 
activities. 

PROJECT  OVERVIEW 

Several  objectives  were  established  in 
the  initial  planning  of  study  activities: 

-  assess  effectiveness  of  the  LSO 
Reverse  Display  for  supporting  LSO 
training 

-  identify  potential  enhancements  to 
the  LSO  Reverse  Display 

-  compare  LSO  Reverse  Display  training 
effectiveness  to  other  device  and 
method  alternatives 

-  delineate  guidance  for  effective  uti¬ 
lization  of  the  LSO  Reverse  Display 
in  terms  of  syllabus  factors  and 
instructor  functions 

-  revise  prior  estimates  of  functional 
performance  capabilities  needed  for 
an  LSO  training  system  as  substanti¬ 
ated  by  research  results. 

The  activities  which  were  planned  to 
support  achievement  included: 

-  a  limited  transfer  of  training  study 

-  surveys  of  the  LSO  community  concern¬ 
ing  the  LSO  Reverse  Display  and 
alternative  LSO  training  devices  and 
methods 

-  observation  of  LSO  Reverse  Display 
operation  in  a  training  context 

-  development  of  syllabii  to  guide  LSO 
Reverse  Display  employment  in  Phase 
II  and  Phase  III  LSO  training. 

During  the  period  of  performance  cov¬ 
ered  by  this  paper  several  activities  actu¬ 
ally  occurred.  A  questionnaire  was  distrib¬ 
uted  within  the  LSO  community  to  elicit 


initial  impressions  of  the  LSO  Reverse  Dis¬ 
play  regarding  its  features  and  its  effective 
utilization.  Project  personnel  spent  time 
on-site  at  NAS  Lemoore  and  NAS  Cecil  Field 
observing  LSO  Reverse  Display  training  opera¬ 
tions  and  collecting  study  data.  A  syllabus 
for  employment  of  the  LSO  Reverse  Display  in 
Phase  II  LSO  training  was  developed. 
Throughout  the  reporting  period  there  was 
extensive  interaction  with  the  LSO  coronunity. 
The  initial  impression  resulting  from  this 
study  is  that  the  LSO  Reverse  Display  shows 
significant  promise  for  effectively  support¬ 
ing  LSO  training. 

THE  LSO  REVERSE  DISPLAY 

The  LSO  Reverse  Display  is  an  LSO  train¬ 
ing  station  which  has  been  added  to  the  A7E 
Night  Carrier  Landing  Trainer  (NCLT),  Device 
2F103.  The  device  is  installed  at  two  Navy 
sites,  NAS  Lemoore,  California,  and  NAS  Cecil 
Field,  Jacksonville,  Florida. 

The  LSO  training  station  is  a  light¬ 
proof  and  sound-proof  enclosure  which  houses 
a  visual  system,  an  LSO  instrument  console, 
normal  LSO  workstation  control  items  and  con¬ 
trol  units  for  training  station  operation. 
There  is  a  separate  instructor  station  which 
serves  both  the  LSO  Reverse  Display  and  the 
NCLT.  This  console  contains  a  single  CRT  for 
viewing  both  the  LSO  and  pilot  approach 
scenes  and  controls  for  operating  the  LSORD. 

From  the  LSO  training  station  (fondly 
called  the  "igloo"),  the  trainee  can  "wave" 
simulated  night  carrier  approaches  flown  by 
the  pilot  in  the  NCLT.  The  LSO  view  includes 
two  CRTs  with  approximately  an  80°  field  of 
view  of  the  carrier  deck  and  horizon.  The 
wall  of  the  training  station  enclosure  has  an 
extended  horizon  line  which  matches  that  por¬ 
trayed  on  the  CRTs.  The  view  of  an  approach¬ 
ing  aircraft  is  maintained  within  the  CRTs 
throughout  the  approach  (including  touchdown 
and  bolter)  by  a  computer  driven  "scene 
rotation"  process.  The  visual  system  can 
depict  carrier  deck  motion.  As  the  aircraft 
approaches  the  carrier,  an  outline  of  the 
aircraft  shape  gradually  appears  on  the  dis¬ 
play.  Engine  sound,  as  heard  from  the  LSO 
platform,  is  provided  and  is  correlated  to 
range  and  to  the  pilot's  throttle  position¬ 
ing.  Background  carrier  deck  noises  are  also 
available.  For  task  interaction  the  trainee 
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has  the  normal  LSO  control  instruments:  radio 
handset,  pickle  and  MOVLAS.  For  instruc¬ 
tional  interaction,  there  are  several  ele¬ 
ments  displayable  to  the  trainee.  There  is  a 
green  crosshair  depicting  optimum  glideslope 
and  lineup,  and  there  is  a  red  crosshair  dur¬ 
ing  rerun  showing  MOVLAS  positioning.  Alpha- 
numerics  which  delineate  approach  results  are 
available  at  the  end  of  an  approach.  During 
rerun,  graphic  plots  of  aircraft  approach 
dynamics  are  available.  The  trainee  has  com¬ 
munications  with  the  NCLT  pilot  and  the 
instructor  console,  through  the  radio  handset 
and  loudspeaker  located  in  the  enclosure. 
Inside  the  enclosure  are  controls  for  oper¬ 
ating  many  features  of  the  device. 

The  instructor  station  at  the  NCLT  con¬ 
sole  has  several  means  for  monitoring  an  LSO 
training  session  including  an  option  for 
viewing  either  the  LSO  or  pilot  scene.  For 
set-up  and  control  of  a  training  situation 
he  has  many  options  available,  some  control¬ 
lable  on  the  console,  some  through  the  com¬ 
puter  system  terminal  and  some  through  com¬ 
municating  with  the  pilot  for  planned 
approach  deviations.  Among  his  most  signif¬ 
icant  instructional  and  debrief  functions  are 
freeze,  approach  rerun  and  control  of  data 
to  be  displayed  to  the  trainee.  For  "pilot¬ 
less"  LSO  training  there  are  ten  canned 
approaches  available. 

STUDY  RESULTS 

SURVEY 

A  questionnaire  entitled  "Evaluation  of 
LSO  Reverse  Display"  was  distributed  to 
approximately  35  LSOs  of  various  skill 
levels  and  within  various  aircraft  communi¬ 
ties.  Only  LSOs  who  had  some  "hands  on" 
familiarity  were  included  in  this  distribu¬ 
tion.  The  paragraphs  below  describe  survey 
resul  ts. 

SURVEY  SAMPLE.  Completed  questionnaires  were 
received  from  17  LSOs.  Two  LSOs  filled  out 
the  questionnaire  twice,  before  and  after 
they  had  some  experience  with  the  LSORD.  Of 
the  17  respondents,  16  identified  themselves 
and  provided  demographic  data.  One  chose  to 
remain  anonymous.  Five  of  the  LSOs  had  pre¬ 
vious  experience  with  the  Night  Carrier  Land¬ 
ing  Trainer  (NCLT),  while  11  had  none.  Only 
2  of  the  LSOs  had  previous  experience  with 
the  LSORD  portion  of  the  NCLT.  Therefore, 

14  (and  possibly  15)  of  the  17  LSOs  were 
responding  to  the  questionnaire  based  on 
'i  mi  ted  experience  with  the  Reverse  Display, 
while  the  LSO  sample  was  limited  in  prior 
exposure  to  the  Reverse  Display,  nearly  all 
had  extensive  LSO  and  pilot  experience.  Six 
were  staff-qualified  LSOs,  two  .were  TRACOM 
qualified,  one  was  WING  qualified,  and  four 
were  SQUADRON  qualified.  Years  of  experience 


as  an  LSO  ranged  from  1  to  11  in  the  sample, 
with  a  median  of  5  years. 

LSORD  FEATURES  RATING.  The  LSOs  were  asked 
to  rate  some  of  the  training  features  of  the 
LSORD,  as  well  as  the  overall  potential  value 
of  the  LSORD  for  training.  Eleven  items  were 
rated  on  a  five-point  scale,  with  1  =  POOR, 

3  =  FAIR,  and  5  =  OUTSTANDING. 

The  “degree  of  realism  of  the  night  car¬ 
rier  approach  scene  from  a  waving  perspec¬ 
tive"  was  given  a  moderate  mean  rating  of 
3.76.  Four  items  received  lower  mean  ratings 
and  six  items  received  higher.  None  of  the 
LSOs  rated  the  scene  realism  as  OUTSTANDING, 
but  94%  rated  it  FAIR  or  better.  Only  one 
LSO  rated  it  below  FAIR. 

The  follow-up  question  on  "night  carrier 
approach  scene  realism"  asked  about  its  "ade¬ 
quacy  for  LSO  training."  All  of  the  LSOs 
rated  it  FAIR  or  better,  and  41%  rated  it 
OUTSTANDING.  The  mean  rating  was  4.24,  the 
second  highest  of  the  eleven  items,  surpassed 
only  by  the  "LSORD  overall  potential  value  to 
LSO  training"  at  4.41. 

The  "degree  of  realism  of  the  LSO  work¬ 
station"  received  a  relatively  low  rating, 
although  94%  gave  it  a  FAIR  or  better.  No 
one  rated  it  OUTSTANDING,  and  one  LSO  rated 
it  below  FAIR.  The  term  "workstation"  in  the 
question  was  general,  so  it  is  difficult  to 
judge  what  aspects  of  the  LSO  workstation 
seemed  unrealistic  to  the  LSOs. 

The  next  three  questions  dealt  wUh  the 
"adequacy  for  training"  of  different  aspects 
of  the  LSORD  workstation.  The  "adequacy  of 
LSO  console  displays  for  LSO  training" 
received  the  lowest  mean  rating  of  the  eleven 
items,  3.47.  However,  there  was  a  wide  range 
of  LSO  responses.  Two  LSOs  rated  it  OUT¬ 
STANDING,  while  three  rated  it  below  FAIR. 
Again,  it  is  not  possible  to  determine  what 
aspects  of  the  console  displays  contributed 
to  the  relatively  low  mean  rating. 

The  “adequacy  of  LSO  controls  for  LSO 
training"  received  the  median  rating.  Five 
items  received  higher  ratings  and  five  lower. 
Six  LSOs  (37%)  rated  the  LSO  controls  OUT¬ 
STANDING.  The  controls  in  the  LSORD  are 
identical  to  those  used  aboard  ship,  includ¬ 
ing  the  LSO  hand  switch  (pickle),  radio, 
and  MOVLAS. 

The  "adequacy  of  sound  simulation  for 
LSO  training"  was  rated  FAIR  or  better  by  all 
of  the  LSOs,  although  none  of  them  rated  it 
OUTSTANDING. 

Two  questions  were  asked  about  the  simu¬ 
lated  aircraft  dynamics.  Their  "realism  from 
a  waving  perspective"  received  a  moderately 


low  rating  of  3.65.  One  LSO  rated  it  below 
FAIR,  but  two  LSOs  rated  it  OUTSTANDING.  By 
contrast,  the  "adequacy  of  the  simulated  air¬ 
craft  dynamics  for  LSO  training"  received  a 
relatively  high  mean  rating  of  4.12. 

It  is  interesting  to  note  that  "simu¬ 
lated  aircraft  dynamics"  questions,  as  well 
as  the  "night  carrier  approach  scene"  ques¬ 
tions,  received  a  relatively  low  rating  for 
"degree  of  realism"  but  a  high  rating  for 
"adequacy  for  LSO  training."  The  relation¬ 
ship  between  simulator  fidelity  and  training 
effectiveness  has  been  the  topic  of  some 
debate  by  training  experts.  Extrapolating 
from  the  LSOs  responses  to  the  survey,  they 
appeared  to  be  stating  that  LSORD  departures 
from  "realism"  were  noticeable,  but  not 
expected  to  detract  from  LSO  training. 

The  adequacy  of  LSO  trainee  evaluation 
was  rated  higher  for  the  instructor  LSO  posi¬ 
tioned  at  the  NCLT  console  than  when  the 
instructor  was  stationed  inside  the  LSORD 
workstation  (the  igloo).  The  reverse  display 
is  available  at  the  NCLT  console,  and  the 
instructor  has  more  controls  available  for 
managing  the  training  process.  Also,  the 
view  of  the  approach  for  the  second  man  (the 
instructor)  in  the  igloo  is  somewhat  awkward 
because  of  the  limited  area  for  proper  head 
and  eye  position.  A  potential  disadvantage 
to  the  instructor  being  located  at  the  NCLT 
console  is  that  communications  with  the  LSO 
trainee  are  by  ICS  rather  than  face  to  face. 

The  final  question  concerned  the  "over¬ 
all  potential  value  of  the  LSORD  to  LSO 
training."  This  item  received  the  highest 
mean  rating,  4.41.  A  rating  of  OUTSTANDING 
was  given  by  53%  of  the  LSOs,  while  35%  rated 
it  GOOD  and  12%  FAIR.  The  LSOs  apparently 
felt  that  the  total  package  was  more  valuable 
for  training  than  the  individual  elements. 
This  overall  rating  was  substantially  higher 
than  any  other  category.  Responses  to  this 
question  may  reflect  the  LSOs  desire  to 
receive  some  training  aid,  since  the  primary 
training  method  currently  is  OJT. 

UTILIZATION  OF  THE  LSORD.  The  LSOs  were 
asked  which  phases  of  LSO  training  could  be 
meaningfully  supported  by  the  Reverse  Dis¬ 
play,  with  the  following  results:  87%  posi¬ 
tive  for  Phase  II,  Field  Carrier  Landing 
Practice  (FCLP),  and  67%  positive  for  Phase 
III,  Carrier  Training  Observation.  A  brief 
description  of  these  LSO  training  phases  is 
provided  in  the  LSO  NATOPS  Manual  (Navy, 
1975). 

When  asked  to  indicate  which  phase  of 
LSO  training  would  be  best  suited  for  the 
LSORD,  20>.  indicated  Phase  I,  40%  Phase  II, 
and  40.X  Phase  III.  The  explanations  which 
accompanied  this  response  emphasized  that 


the  LSO  trainee  who  would  roost  benefit  from 
the  LSORD  would  be  one  who  had  seen  night 
carrier  operations.  The  LSOs  selected  one  or 
more  levels  of  LSO  experience  for  which  the 
LSORD  would  be  suitable  for  supplemental 
training.  The  outcome  corroborates  the 
results  from  the  previous  question.  The 
highest  category  was  "slight  experience 
aboard  ship."  The  second  place  category  was 
interesting,  in  that  a  completely  different 
application  of  the  LSORD  was  implied  - 
refresher  training  rather  than  initial  train¬ 
ing.  The  second  category  was  "Wing  Qual 
returning  from  non-LSO  tour  of  duty." 

Another  issue  in  utilization  of  the 
LSORD  is  the  level  of  pilot  skill  appropriate 
for  flying  the  NCLT  when  conducting  LSO 
training.  Four  categories  of  pilots  were 
suggested,  and  the  associated  percentage  of 
LSOs  who  favored  each  was  as  follows:  RAG 
student  pilot,  8%;  squadron  nugget,  31%; 
experienced  pilot  (.non-LSO),  15%;  and  LSO, 

46%.  When  the  LSOs  were  asked  to  identify 
their  second  choice  for  NCLT  pilot,  “experi¬ 
enced  pilot"  was  highest  with  58%,  followed 
by  "squadron  nugget"  with  25%.  These  results 
suggest  some  scheduling  and  utilization  dif¬ 
ficulties  for  the  LSORD.  The  pilot  of  last 
choice  for  LSO  training  was  the  RAG  student 
pilot,  but  the  NCLT  is  used  primari ly  for 
training  RAG  student  pilots. 

Several  items  were  frequently  noted  when 
asked  about  "system  characteristics  and/or 
capabilities. ..particularly  valuable  to  LSO 
training."  Pitching  deck  simulation  was  most 
frequently  noted.  MOVLAS  and  "pickle"  time 
in  an  adequately  simulated  night  carrier 
landing  environment  were  also  items  of  sig¬ 
nificant  note. 

Several  items  were  frequently  noted  when 
asked  about  needed  improvements  for  LSO 
training.  Overall  perceptual  difficulties 
for  the  "in  close"  and  "at  the  ramp"  portions 
of  the  approach,  and  nose  altitude  and  lineup 
perception  throughout,  were  most  frequently 
noted.  Incorporation  of  additional  aircraft 
types  into  the  simulation  was  a  frequent  sug¬ 
gestion  for  improving  the  system. 

Notable  comments  concerning  optimum  uti¬ 
lization  seem  to  emphasize  frequent  utiliza¬ 
tion,  use  of  the  device  between  carrier  ops 
periods  (CQ,  type  training)  and  utilization 
by  the  enti re  LSO  community  whenever  possible. 

LSO  PERFORMANCE  MEASURES.  The  LSOs  were 
asked  to  rate  23  potential  measures  of  LSO 
performance.  The  list  of  candidate  measures 
of  LSO  performance  were  not  specifically 
related  to  the  LSORD.  The  underlying  strat¬ 
egy  was  to  allow  LSOs  to  identify  reasonable 
measures  of  LSO  job  performance,  most  of  which 
can  be  obtained  within  the  NCLT/LSORD  system. 


Ratings  were  given  on  a  five-point  scale 
from  1  =  definitely  not  indicative  of  LSO 
performance  through  5  =  definitely  indicative. 
The  three  highest  ratings  were  for  measures 
which  are  dependent  solely  on  the  LSO  and  not 
on  a  combination  of  LSO  and  pilot  performance. 
"Correctness  and  timeliness  of  LSO  calls" 
received  the  highest  rating,  with  94%  of  the 
LSOs  surveyed  rating  it  DEFINITELY  INDICATIVE. 
The  next  two  candidate  performance  measures 
with  highest  ratings  were  "correct  recogni¬ 
tion  of  glideslope,  lineup,  and  AOA  devia¬ 
tions,"  and  "MOVLAS  positioning  accuracy." 

The  remaining  20  items  all  related  to 
aircraft  flight  parameters  or  landing  out¬ 
come.  These  variables  all  are  composite  of 
the  performance  of  the  pilot,  aircraft,  and 
LSO.  Therefore,  the  measures  tend  to  be 
influenced  by  the  LSO,  but  not  directly  con¬ 
trolled  by  him. 

The  highest  rating  for  this  type  of 
measure  was  "ramp  strike  rate,"  followed  by 
"sink  rate  at  touchdown,"  "glideslope  devia¬ 
tion  during  the  approach,"  "boarding  rate," 
and  "line-up  deviations."  The  lowest  ratings 
were  given  for  measures  that  most  directly 
relate  to  pilot  technique  -  "rough  nose"  and 
"rough  power."  These  measures  were  rated,  at 
best,  as  "possibly  indicative"  of  LSO 
performance. 

A  formal  definition  of  the  highest  rated 
performance  measure,  "correctness  and  timeli¬ 
ness  of  LSO  calls"  is  currently  being  devel¬ 
oped  in  another  NTEC  project  by  Human  Per¬ 
formance  Research,  Inc.  (Borden  and  McCauley, 
1978). 

Unfortunately,  from  the  viewpoint  of 
ease  of  performance  measurement,  it  is  the 
composite  (pilot/aircraft/LSO)  measures  that 
are  easiest  to  obtain  from  the  NCLT  software. 
"Correctness"  of  LSO  calls  currently  is  a 
subjective  judgment,  not  amenable  to  measure¬ 
ment  within  the  NCLT/LSORD  system. 

OBSERVATION  ACTIVITY 

On-site  observation  of  LSO  Reverse  Dis¬ 
play  operation  involved  six  visits.  Four 
visits  were  to  NAS  Lemoore  and  the  other  two 
were  to  NAS  Cecil  Field.  The  first  three 
visits  were  oriented  to  familiarization  with 
the  features  and  operation  of  the  system. 

The  last  three  were  oriented  to  observing 
the  operation  of  the  system  in  a  training 
context.  At  NAS  Lemoore  (two  visits)  Phase 
II  training  was  being  conducted.  Phase  III 
training  was  being  conducted  at  NAS  Cecil 
Field  (one  visit).  The  trainees  involved 
with  the  LSO  Reverse  Display  had  entry 
levels  varying  from  inexperienced  to  squadron 
qualification  with  night  shipboard  waving 
experience.  All  trainees  were  from  the  A7 


comnunity  except  one,  who  was  from  the  S-3 
community.  Two  LSO  Instructors  were  involved. 
One  was  from  the  A7  Fleet  Readiness  squadron; 
the  other  was  an  Air  Wing  LSO.  Both  instruc¬ 
tors  were  very  highly  skilled  LSOs  and  demon¬ 
strated  a  high  degree  of  motivation  and  con¬ 
scientiousness  in  their  instructional  duties. 

The  observation  effort  also  included 
frequent  interrogation  of  LSO  instructors  and 
trainees  concerning  their  impressions  of  var¬ 
ious  features,  instructor  techniques, 
instructional  strategies  and  conduct  of 
training  sessions.  Information  gathered 
during  observation  included  frequency  of  fea¬ 
ture  utilization,  opinions  of  feature  effec¬ 
tiveness,  instructor  techniques  used,  proce¬ 
dures  followed  during  training  sessions, 
difficulties  in  system  operation  and  instruc¬ 
tor  evaluation  of  trainee  performance. 

Although  both  instructors  appeared  to 
perform  their  duties  with  equal  effective¬ 
ness,  some  interesting  differences  were 
noted.  One  instructor  preferred  to  monitor 
the  LSO  view  most  of  the  time,  the  other  pre¬ 
ferred  the  pilot  view.  One  used  approach 
results  data  and  approach  dynamics  rerun 
plots  frequently  for  helping  the  trainee 
learn  the  perceptual  aspects  of  the  task. 

The  other  preferred  to  rely  on  rerun  and 
frequent  freeze  for  pointing  out  approach 
deviations.  One  placed  heavy  emphasis  on 
immediate  recall  of  the  approach  for  grading 
pilot  performance.  The  other  focused  pri¬ 
marily  on  the  waving  of  the  approach. 

SYLLABUS  DEVELOPMENT 

The  initial  activity  in  Phase  II  sylla¬ 
bus  development  involved  the  identification 
of  tasks  associated  with  the  conduct  of  FCLP. 
The  task  listing  presented  in  an  earlier  NTEC 
report  (Hooks  et  al.,  1978),  guided  this 
effort.  These  tasks  were  then  correlated  to 
the  syllabus  design  rationale  and  the  sylla¬ 
bus  from  that  report,  as  well  as  the  Phase  II 
training  guidelines  of  LSO  NATOPS  (Navy, 

1975).  This  provided  a  tentative  sequencing 
of  learning  activities  for  the  syllabus.  The 
strategy  of  dividing  the  syllabus  into  two 
learning  stages  also  evolved  out  of  this 
process.  Separate  orientations  of  basic  wav¬ 
ing  skills  and  pilot  refresher  training 
skills  enables  the  trainee  to  acquire  confi¬ 
dence  in  the  perceptual  and  decision-making 
aspects  of  his  performance  prior  to  concen¬ 
tration  on  fine  tuning  his  critique  of  pilot 
performance  for  instructional  purposes.  This 
strategy  also  emphasizes  concentrated  "hands 
on"  waving  experience  in  the  initial  stage  of 
learning.  A  review  of  the  Phase  II  syllabus, 
developed  by  LT  Bullard  of  VA-122,  and  the 
information  gathered  from  survey  and  observa¬ 
tion  activities  provided  additional  refine¬ 
ments  to  the  sequence.  It  also  guided  the 
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final  syllabus  mix  of  available  media  (FCLP, 
LSO  Reverse  Display,  lecture). 

The  reconmended  syllabus  provides  stand¬ 
ardized  guidance  for  learning  activity 
sequencing  and  the  integration  of  various 
media.  More  detailed  syllabus  guidance  does 
not  appear  feasible  at  this  time  because 
there  will  be  differences  in  the  learning 
rates  for  different  LSO  task  components  among 
trainees.  A  better  understanding  of  LSO  job 
behaviors,  performance  criteria  and  LSO 
learning  processes  is  required  before  more 
specific  syllabus  guidance  can  be  formulated. 

CONCLUSIONS 

THE  LSO  REVERSE  DISPLAY 

The  LSO  Reverse  Display  has  demonstrated 
the  potential  to  be  a  very  effective  training 
device  for  the  A-7  LSO  community.  As  a  mini¬ 
mum,  simulation  of  the  night  carrier  landing 
environment  and  an  approaching  A-7  aircraft 
enables  an  LSO  trainee  to  perform  a  consider¬ 
able  portion  of  the  LSO  waving  task.  The 
highlights  of  the  LSO  Reverse  Display  are 
simulation  of  pitching  deck  conditions  and 
provisions  for  using  the  MOVLAS  to  wave 
approaches,  two  aspects  of  the  LSO  job  which 
are  inadequately  addressed  in  the  existing 
OJT  program.  Its  capability  for  instruc¬ 
tional  control  of  carrier  landing  situations 
can  provide  training  benefit  to  the  naive 
LSO  trainee  as  well  as  a  trainee  approaching 
Wing  qualification.  It  also  shows  promise 
as  an  aid  to  refresher  training  for  those 
returning  from  non-LSO  tours  of  duty. 

The  LSO  Reverse  Display  does,  however, 
have  some  limitations  of  varying  signifi¬ 
cance.  One  is  its  obvious  orientation  and 
convenient  location  to  the  A-7  LSO  community, 
thus  minimizing  its  impact  on  other  Navy  LSO 
training.  The  requirement  for  a  pilot  in  the 
NCLT  impacts  personnel  support  for  LSO  train¬ 
ing.  There  are  some  operating  inefficiencies 
associated  with  the  conduct  of  LSO  training 
in  the  device.  There  are  some  perceptual 
difficulties  for  the  trainee  in  the  final 
portion  of  the  approach. 

The  identified  limitations  do  not  over¬ 
ride  the  benefits  arising  from  integration 
of  this  device  into  LSO  training.  The  key 
to  its  value  is  effective  and  conscientious 
utilization. 

LSO  REVERSE  DISPLAY  FEATURES.  Some  of  the 
more  significant  conclusions  regarding  fea¬ 
tures  of  the  LSO  Reverse  Display  are  pre¬ 
sented  below. 

For  the  student  station  several  items 
are  noteworthy.  The  carrier  approach  back¬ 
ground  scene  provides  adequate  fidelity  for 


training.  The  clear/foul  deck  light  should 
be  more  prominent.  Deck  motion  simulation  is 
an  exceptionally  valuable  feature  for  LSO 
training.  The  A7  visual  simulation  provides 
adequate  fidelity  for  training.  Frequently 
there  is  a  convergence  problem  with  aircraft 
lighting.  During  scene  rotation,  aircraft 
flight  dynamics  do  not  maintain  fidelity: 
aircraft  looks  higher  and  flatter  at  the  ramp 
than  it  actually  is;  also  appears  to  get 
smaller  than  it  should.  Nose  down  pitch 
attitude  and  lineup  are  difficult  to  per¬ 
ceive.  Absence  of  waveoff  repeater  lights  on 
the  LSO  instrument  console  is  an  irritant  to 
the  conduct  of  training  sessions.  Audio  sim¬ 
ulation  for  the  approaching  aircraft  has  ade¬ 
quate  fidelity.  The  controls  provided  in  the 
student  station  are  adequate  for  training. 

The  MOVLAS  is  actual  equipment  and  is  one  of 
the  most  valuable  features  for  LSO  training. 
Crosshair  is  a  valuable  cueing  aid  but 
trainee  must  be  clearly  aware  of  its  proper 
relationship  to  the  aircraft  for  optimum 
approach  positioning.  Also  the  trainee  must 
select  his  own  crosshair,  a  difficult  process 
mentioned  later.  Crosshair  for  MOVLAS  posi¬ 
tioning  during  rerun  is  a  valuable  debriefing 
aid.  Approach  results  data  ("MONITOR")  has 
limited  value  to  LSO  trainee  due  to  problem 
with  perception  during  scene  rotation  (men¬ 
tioned  earlier)  and  delaying  rerun  while  data 
is  displayed.  Approach  dynamics  plots 
("SCORE")  provide  useful  pilot  error  analysis 
data  during  rerun  but  value  is  slightly  less¬ 
ened  because  of  cluttered  view  of  approach. 
Coimuni cations  between  trainee  and  instructor 
through  the  intercom  are  frequently  garbled. 
The  cause  appears  to  be  the  acoustic  quality 
of  the  loudspeaker  in  the  "igloo."  Proper 
eye  positioning  within  available  viewing 
volume  is  initially  difficult  to  attain. 
Effective  positioning  of  two  heads  within  the 
viewing  volume  is  difficult  to  achieve,  thus 
constraining  "over  the  shoulder"  training. 
Most  of  the  controls  for  operating  system 
features  are  enclosed  on  a  pedestal  mounting 
behind  the  LSO  trainee.  To  activate  con¬ 
trols,  such  as  crosshair  selection,  requires 
moving  from  the  optimum  viewing  position, 
raising  the  pedestal  cover  and  activating  the 
desired  switch  (two  hands  needed).  This 
process  is  very  cumbersome.  The  cover  for 
station  controls  must  remain  closed  during 
approaches  because  of  significant  glare  gen¬ 
erated  on  the  CRT  optics  by  the  brightly 
lighted  push-tile  switches. 

For  the  instructor  station  several  items 
are  noteworthy.  The  instructor  has  adequate 
information  for  monitoring  training  situa¬ 
tions  and  trainee  performance  through  CRT 
display,  notation  of  control  selections, 
voice  communications  and  aircraft  instru¬ 
ments.  Placement  of  controls  for  changing 
between  pilot  and  LSO  CRT  views  is  ineffi¬ 
ciently  located  at  far  end  of  console.  The 
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instructor  must  turn  his  crosshair  off  to 
determine  whether  trainee  has  crosshair 
selected.  The  instructor  cannot  monitor 
MOVLAS  positioning  at  the  console  during  an 
approach  (only  during  rerun).  However,  he 
can  monitor  meatball  positioning  in  the  pilot 
view  and  optimum  glideslope  on  the  X-V  plot¬ 
ter.  The  instructor  cannot  monitor  trainee 
waveoff  light  activation  at  the  console.  The 
instructor  cannot  activate  the  student  sta¬ 
tion  crosshair  from  the  console.  There  are 
insufficient  canned  approaches  available  in 
the  system  for  meaningful  "pilotless" 
instruction.  Rerun  of  a  recorded  approach 
is  a  valuable  training  feature  of  the  system. 
Availability  of  the  MOVLAS  positioning  cross¬ 
hair  during  rerun  is  a  valuable  debriefing 
aid. 

LSO  Rb VERSE  DISPLAY  UTILIZATION.  The  LSO 
Reverse  Display  appears  to  have  the  capabil¬ 
ity  to  support  many  levels  of  LSO  training, 
some  levels  more  significantly  than  others. 

If  effectively  employed,  it  should  decrease 
calendar  time  required  for  LSO  skill  acquisi¬ 
tion  and  provide  significantly  more  "hands 
on"  experience  with  more  portions  of  the  LSO 
task  than  are  currently  available  in  OJT. 

The  comments  below  apply  to  training  in  the 
total  LSO  community  (not  just  A-7).  As  men¬ 
tioned  earlier,  limited  access  to  the  LSO 
Reverse  Display  will  limit  the  benefit  to 
LSOs  outside  the  A-7  community. 

a.  Phase  I  LSO  Training.  As  a  part  of 
the  LSO  Phase  l  School  curriculum, 
the  LSO  Reverse  Display  could  pro¬ 
vide  instructional ly  controlled 
familiarization  with  the  night  car¬ 
rier  landing  environment  and  limited 
"hands  on"  familiarization  with  the 
waving  task. 

b.  Phase  II  LSO  Training.  The  LSO 
Reverse  Display  can  enable  earlier 
"hands  on"  waving  experience  than  is 
now  possible  in  OJT.  It  can  also 
enable  incremented  acquisition  of 
components  of  the  basic  waving  skill 
as  well  as  extensive  integrated 
practice.  It  can  also  support  the 
learning  of  pilot  grading,  diagnosis 
and  debrief  skills.  MOVLAS  and  LSO 
talkdown  are  Phase  II  skills  for 
which  the  LSO  Reverse  Display  is 
particularly  suited.  A  Phase  II 
training  program  integrating  the  use 
of  FCLP  and  LSO  Reverse  Display  can 
probably  graduate  a  trainee  who  is 
much  better  prepared  for  Phase  III 
than  was  ever  possible  before. 

c.  Phase  II  Syllabus.  One  of  the  prime 
assets  of  the  LSO  Reverse  Display  in 
support  of  LSO  training  is  the  provi¬ 
sion  for  safe,  "hands  on"  waving 


experience  at  the  outset  of  Phase  II 
training.  Another  prime  asset  is  the 
instructional  control  available 
through  such  features  as  freeze, 
rerun  and  crosshair.  Of  the  situa¬ 
tional  features  available,  MOVLAS  and 
LSO  talkdown  conditions  are  particu¬ 
larly  appropriate  to  Phase  II  train¬ 
ing.  An  effective  syllabus  can 
employ  these  and  other  features  of 
the  Reverse  Display,  in  conjunction 
with  the  FCLP  environment,  to  guide 
the  trainee  through  the  completion  of 
Phase  II  training.  The  syllabus 
designed  as  a  part  of  this  project 
provides  a  topical  sequencing  and 
resource  allocation  plan  for  Phase  II 
training.  It  promotes  incremental 
acquisition  of  field  qualification 
skills  in  two  stag  s  of  orientation, 
waving  and  pilot  training.  The  syl¬ 
labus  is  of  fixed  design.  Adaptive 
instructional  control,  based  on 
learner  differences  in  acquiring  LSO 
skills,  can  be  implemented  by  the 
instructor  using  the  Reverse  Display 
for:  remedial  training  sessions, 
additional  practice,  and  accelerated 
introduction  of  new  topics  during 
training  sessions.  It  is  expected 
that  trainee  performance  and  learnim 
rate  while  in  the  Reverse  Display 
will  be  significantly  enhanced  if  he 
has  observed  night  operations  aboard 
Ship. 

d.  Phase  III  LSO  Training.  The  LSO 
Reverse  Display  appears  best  suited 
to  support  Phase  III  training  because 
of  its  simulation  of  a  considerable 
portion  of  the  night  carrier  landing 
environment.  Extensive  use  of  the 
LSO  Reverse  Display  between  CQ  and 
deployment  work-up  periods  looks  like 
the  most  effective  timing  of  the 
device  in  Phase  III.  The  increased 
waving  experience  available  through 
the  LSO  Reverse  Display  should  accel¬ 
erate  skill  acquisition.  "Hands  on" 
experience  with  pitching  deck,  MOVLAS 
and  aircraft  emergency  situations 
should  produce  a  more  highly  skilled 
Wing  qualification  LSO  than  is  now 
possible. 

e.  Proficiency  and  Refresher  LSO  Train¬ 
ing.  The  simulation  fidelity  appears 
adequate  to  support  limited  mainte¬ 
nance  of  skill  proficiency  during 
extended  time  periods  between  deploy¬ 
ments.  The  Reverse  Display  appears 
even  more  promising  for  its  potential 
to  support  refresher  training  for 
LSOs  returning  from  non-LSO  tours  of 
duty. 


f.  Potential  Limitations  to  LSO  Reverse 
Display  Effectiveness.  Several  fac¬ 
tors  could  limit  effectiveness  of  the 
LSO  Reverse  Display  in  LSO  training. 
Most  are  not  unique  to  the  LSO 
Reverse  Display.  A  shortage  of 
instructor  LSOs  can  significantly 
reduce  device  utilization  and  effec¬ 
tiveness.  Heavy  loading  of  pilot 
training  in  the  NCLT  can  limit  avail¬ 
ability  of  LSO  Reverse  Display  train¬ 
ing  time.  Limitations  in  trainee 
availability  for  LSO  training  due  to 
flying  duties,  school  attendance, 
other  job  responsibilities,  etc.,  is 
also  a  factor.  Negative  attitudes 
of  instructor,  trainee  and  supervi¬ 
sory  personnel  toward  the  LSO  Reverse 
Ois  ay  can  impact  its  utilization 
rat.  and  negate  its  effectiveness. 
Inattention  to  quality  control  of  the 
training  syllabus  can  cause  degrada¬ 
tion  of  LSO  Reverse  Display  training 
effecti  veness. 

LSO  TRAINING  SYSTEM  CONCEPTS 

A  critical  shortage  of  skilled  LSO  con¬ 
tinues  to  plague  the  Navy.  Excessively  long 
training  time,  reduced  carrier  operations 
and  pilot  retention  remain  as  the  primary 
causal  factors.  The  lower  carrier  operations 
activity  level  experienced  by  west  coast 
fleet  units  makes  their  problem  more  severe 
than  for  east  coast  units.  A  recent  slowing 
of  airline  hiring  may  result  in  an  easing  of 
retention  problems.  However,  it  will  prob¬ 
ably  not  have  immediate  impact  on  the  LSO 
problem.  The  long  term  impact  may  also  be 
minimal  since  the  duration  of  the  airline 
hiring  slowdown  is  unpredictable. 

The  continuation  of  the  LSO  shortage 
confirms  the  need  for  measures  to  accelerate 
LSO  skill  acquisition.  The  promise  demon¬ 
strated  by  the  LSO  Reverse  Display  confirms 
the  likelihood  that  a  training  system  (or 
systems)  can  help  accelerate  LSO  training  and 
can  provide  enhanced  skill  levels  for  some 
portions  of  the  LSO  task  (MOVLAS,  pitching 
deck,  etc.). 

There  appears  to  be  a  tightening  of 
travel  funds  in  the  Navy.  This  has  a  nega¬ 
tive  impact  on  accessibility  to  the  LSO 
Reverse  Display  by  LSOs  and  trainees  not 
located  at  NAS  Cecil  Field  and  NAS  Lemoore. 
The  need  for  location  of  training  systems 
convenient  to  the  total  LSO  community 
appears  even  more  significant. 

Another  factor  which  must  continue  to  be 
considered  is  the  cost  savings  which  are 
probable  with  improved  LSO  training.  If  an 
improvement  (such  as  an  LSO  training  system) 
prevents  one  fatal  ramp  strike  accident. 


there  is  a  multi-million  dollar  savings  to 
the  Navy.  Costs  of  aircraft  are  in  the 
tens  of  millions  of  dollars  and  the  costs 
of  aircrew  training  are  approaching  a  million 
dollars.  An  aircrew  fatality  may  be  an  even 
greater  loss  in  view  of  existing  retention 
problems. 

Evaluation  of  the  LSO  Reverse  Display 
has  provided  an  opportunity  to  draw  tentative 
conclusions  regarding  the  LSO  training  system 
functional  concepts  recommended  earlier 
(Hooks  et  al.,  1978).  The  LSO  Reverse  Dis¬ 
play  has  functional  similarity  to  the  system 
described  in  that  report  with  the  exceptions 
of  automated  adaptive  control  and  performance 
evaluation.  The  LSO  Reverse  Display  has 
shown  such  promising  potential  to  LSO  train¬ 
ing,  there  is  strong  support  for  the  notion 
that  a  universal  training  system  can  signifi¬ 
cantly  ease  the  LSO  training  problem.  The 
following  paragraphs  provide  discussion  of 
several  functional  aspects  of  an  LSO  train¬ 
ing  system. 

Computer  generated  imagery  for  approach¬ 
ing  aircraft  and  background  scene  simulation 
appears  adequate  for  training  from  both 
fidelity  and  mani  pul  ability  perspectives. 
Field  utilization  seems  to  confirm  the 
earlier  notion  that  the  night  environment  is 
required,  but  a  day  scene  is  of  questionable 
cost-effectiveness.  There  still  appears  to 
be  a  strong  requirement  for  simulation  of 
multiple  aircraft  types  to  satisfy  total  LSO 
community  training  needs.  LSO  commentary 
regarding  scene  rotation  problems  in  the  LSO 
Reverse  Display  seem  to  support  the  need  for 
visual  scene  field  of  view  coverage  of  the 
landing  area.  The  value  of  the  LSO  Reverse 
Display  crosshair  verifies  the  need  for  arti¬ 
ficial  cueing  to  aid  the  trainee  in  learning 
to  perceive  approach  parameter  variations. 
Providing  scene  highlighting  and  annotation, 
pilot  view  of  approach  and  dynamic  decision- 
aiding  "zones"  appear  to  be  promising 
instruction  effects. 

Potential  pilot  availability  problems, 
instructional  presentation  efficiency  con¬ 
siderations  and  the  potential  value  of  lim¬ 
ited  instructorless  LSO  training  support  the 
notion  of  an  automated  pilot/aircraft 
("pilotless"  system)  capability. 

The  instructional  requirements  for 
wind-over-deck,  lens  setting,  hook-to-ramp 
and  MOVLAS  repeater  displays  still  appear 
strong.  The  need  for  a  PLAT  display  now 
appears  controversial  except  as  a  workstation 
fidelity  issue.  New  knowledge  that  HUD  and 
CLASS  are  likely  to  be  incorporated  into  the 
LSO  workstation  makes  them  probable  candi¬ 
dates  for  the  training  system.  The  training 
system  would  be  an  excellent  vehicle  for 
learning  how  to  use  them. 
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Implementation  of  pitching  deck  cues 
through  the  visual  system  alone  appears  com¬ 
pletely  adequate  for  an  LSO  training  system 
which  employs  an  infinity  optics  visual  sys¬ 
tem  interface.  No  conclusion  can  yet  be 
drawn  for  a  large  screen  projection  type  of 
installation. 

Automated  instructional  guidance  and 
evaluative  feedback  would  seem  to  increase 
training  session  efficiency  under  instructor 
control,  and  would  be  necessary  for  an 
"instructorless"  mode  of  system  utilization 
(which  looks  promising  in  view  of  LSO  job 
loading  and  LSO  shortages). 

The  cost-effectiveness  of  sophisticated 
automated,  adaptive  control  remains  unknown 
without  a  better  understanding  of  the  feasi¬ 
bility  of  modelling  LSO  job  behavior  and 
learning  processes.  However,  evaluation  of 
the  LSO  Reverse  Display  has  shed  some  light 
on  limited  levels  of  automated  adaptive  con¬ 
trol  and  instructor  interface  functions. 
Limited  adaptive  control  automation  for  task 
and  strategy  selection  within  a  prescribed 
syllabus  could  be  initiated  through  manual 
performance  evaluation  inputs  from  the 
instructor.  This  could  alleviate  some  set-up 
and  instructional  decision  loading  on  the 
instructor  and  improve  training  session 
efficiency. 

The  LSO  community  feels  that  evaluation 
and  instructor  control  of  learning  situations 
are  best  accomplished  through  instructor 
judgment.  A  promising  automated  control 
application  might  be  focused  on  critical  wav¬ 
ing  situations  for  which  performance  criteria 
could  be  well-defined.  A  data  collection  and 
storage  subsystem  of  student  task  performance 
and  scenario  experience  data  could  provide 
the  basis  for  subsequent  development  of  a 
sophisticated  and  effective  adaptive  logic. 

The  need  for  co-location  of  the  instruc¬ 
tor  with  the  trainee  appears  to  have  less¬ 
ened,  based  on  LSO  Reverse  Display  evaluation 
results.  Subjective  evaluation  appears  very 
adequately  supported  by  monitoring  of  pilot 
and  LSO  views,  voice  communications  and 
trainee  control  activations.  Uncertainty 
reqarding  display  of  objective  performance 
evaluation  support  data  (to  the  instructor) 
continues  because  of  the  lack  of  an  objective 
performance  evaluation  method.  Automation 
support  to  the  instructor  for  scenario  set-up 
would  help  training  session  conduct  effi¬ 
ciency.  It  could  effectively  support  trainee 
utilization  of  the  system  in  "instructorless" 
si tuations. 

As  mentioned  earlier,  little  has  yet 
been  learned  concerning  the  implementation  of 
automated  performance  evaluation  support. 

The  correctness  and  timeliness  of  LSO  calls 


and  control  activations  continue  to  be  sup¬ 
ported  as  the  key  elements  of  LSO  performance 
quality.  Reliable  modelling  of  these  ele¬ 
ments  and  other  candidate  Indicators  for 
evaluation  purposes  remains  uncertain,  except 
possibly  in  the  case  of  well-defined,  criti¬ 
cal  situations  as  mentioned  earlier. 

PROTOTYPE  AUTOMATED  LSO  TRAINING  SYSTEM 

As  discussed  earlier,  some  uncertainty 
still  remains  concerning  the  functional  char¬ 
acteristics  of  an  automated  LSO  training  sys¬ 
tem.  However,  it  appears  that  there  will  be 
sufficient  information  available  from  NTEC- 
sponsored  projects  during  1980  to  proceed 
with  the  development  of  a  prototype  system. 
The  need  for  timely  aid  to  the  LSO  training 
problem  is  another  important  consideration. 
Attempts  to  empirically  resolve  all  system 
design  uncertainty  would  result  in  excessive 
delays.  A  prototype  system  could  serve  as 
both  an  operational  training  system  and  an 
experimental  tool  to  provide  more  definitive 
guidance  for  the  characteristics  of  future 
LSO  training  systems. 

RECOMMENDATIONS 
LSO  REVERSE  DISPLAY  FEATURES 

Recommendations  for  enhancement  of  LSO 
Reverse  Display  features  are  separated  into 
two  groupings:  modifications  which  appear  to 
be  relatively  simple  and/or  inexpensive,  and 
those  which  appear  complex  and/or  require 
signi:  .ant  analytical  attention. 

SIMPLE  ENHANCEMENTS.  The  most  promising 
improvement  to  the  system  would  be  incorpo¬ 
ration  of  a  capability  of  storing  and  pre¬ 
senting  a  larger  number  of  canned  approaches. 
The  red  MOVLAS  positioning  crosshair  should 
be  optionally  displayable  in  real-time  (dur¬ 
ing  an  original  approach)  at  both  the 
instructor  and  trainee  stations.  There 
should  also  be  an  indication  of  trainee  wave- 
off  light  activation  at  both  the  instructor 
and  trainee  stations.  The  clear/foul  deck 
light  should  be  more  prominent  on  the  CRT 
display.  The  sound  quality  of  the  intercom 
system  should  be  improved.  There  should  be 
control  and  display  of  trainee  crosshair 
selection  at  the  instructor  console.  The 
pilot  view  of  the  approach  should  be  option¬ 
ally  available  for  display  at  the  trainee 
station. 

COMPLEX  ENHANCEMENTS.  There  are  two  very 
significant  items  which  require  attention. 

One  is  investigation  and  solution  of  the  per¬ 
ceptual  difficulties  associated  with  the 
final  portion  of  the  approach  ("in  close," 

"at  the  ramp"  and  during  scene  rotation). 

The  other  is  the  difficulty  in  perceiving 
nose  down  attitudes  during  the  approach.  An 
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investigation  should  be  made  into  the  modifi¬ 
cation  of  PMS  to  provide  LSO  performance 
evaluation  support.  Automation  of  pilot  and 
aircraft  dynamics  (providing  a  "pilotless" 
training  capability)  is  very  worthy  of  atten¬ 
tion.  simulation  and  display  of  additional 
aircraft  types  is  also  worthy  of  attention. 
Implementation  of  automated  trainee  perform¬ 
ance  evaluation,  based  on  correctness  and 
timeliness  of  voice  calls  and  control  acti¬ 
vations  appears  worthy  of  attention. 

LSO  REVERSE  DISPLAY  UTILIZATION 

Recommendations  for  LSO  Reverse  Display 
utilization  are  oriented  toward  confirmation 
of  the  preliminary  study  conclusions  pre¬ 
sented  earlier.  As  a  basis  for  confirmation, 
the  LSO  coranunity  must  take  advantage  of 
every  opportunity  to  exercise  the  LSO  Reverse 
Display  in  training.  Currently,  there  is  an 
insufficient  sample  of  the  LSO  community  with 
any  significant  experience  at  using  the 
device.  If  future  studies  are  to  be  fruit¬ 
ful,  this  sample  must  be  expanded. 

It  is  recommended  that  the  Phase  II  LSO 
training  syllabus  developed  in  this  project 
receive  timely  review  by  the  LSO  community 
and  that  this  syllabus,  or  modifications  of 
it,  be  implemented  by  appropriate  supervi¬ 
sory  LSOs  (most  likely  air  wing  LSOs).  The 
sooner  this  occurs,  the  sooner  the  LSO  com¬ 
munity  can  receive  the  benefits  of  formalized 
Phase  II  training  and  the  sooner  the  syllabus 
can  be  evaluated  in  an  operational  context. 

During  utilization  of  the  LSO  Reverse 
Display  particular  attention  should  be  paid 
to  effective  employment  of  the  various  fea¬ 
tures  of  the  device.  The  conclusions  pre¬ 
sented  earlier  provide  a  baseline  for  feature 
utilization.  However,  extensive  exercise  of 
the  device  should  stimulate  additional  com¬ 
mentary  concerning  better  instructional  tech¬ 
niques  and  device  operation  procedures. 

LSO  TRAINING  SYSTEM  CONCEPTS 

The  promise  demonstrated  by  the  LSO 
Reverse  Display  is  sufficient  to  warrant  a 
recommendation  for  procurement  of  an 


experimental  prototype  automated  LSO  training 
system.  This  system  should  have  character¬ 
istics  suitable  to  training  within  the  total 
LSO  community  and  should  have  the  capability 
to  collect  LSO  performance  data.  The  design 
of  this  system  should  obviously  be  based  on 
the  final  results  of  this  and  other  related 
NTEC  projects.  Special  attention  should  be 
given  to  the  selection  of  visual  subsystem 
capabilities,  in  view  of  the  visual  percep¬ 
tion  problems  observed  in  the  LSO  Reverse 
Display. 

SUMMARY 

Procurement  of  the  LSO  Reverse  Display 
has  provided  two  significant  benefits  to  the 
LSO  community.  Firstly,  it  provides  LSO 
training  support  to  LSOs  in  the  A7  community. 
Secondly,  it  has  provided  an  opportunity  for 
field  evaluation  of  promising  automated  LSO 
training  system  concepts.  Both  benefits  are 
likely  to  support  the  easing  of  LSO  training 
problems.  The  A7  community  has  already 
started  realizing  its  benefits  in  the  upgrad¬ 
ing  of  its  LSO  training  program.  In  the  near 
future,  the  Navy  should  be  able  to  utilize 
the  results  of  this  study  to  support  deci¬ 
sions  concerning  the  procurement  of  future 
LSO  training  systems. 
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INTRODUCTION 

The  Navy's  Landing  Signal  Officer  (LSO) 
plays  a  critical  role  in  the  safety  and 
effectiveness  of  carrier  operations.  There 
currently  exists  a  shortage  of  skilled  LSO 
personnel  that  approaches  the  crisis  level. 
Aggravating  the  problem  is  the  fact  that  it 
takes  a  long  time  for  the  existing  training 
program  to  develop  an  LSO  to  a  productive 
skill  level.  The  Human  Factors  Laboratory 
of  NTEC  is  sponsoring  research  efforts 
pointed  toward  determining  measures  which 
can  cost-effectively  reduce  LSO  training 
time.  One  such  effort,  which  is  currently 
underway,  is  to  develop  a  laboratory  LSO 
training  system  for  research  into  the  appli¬ 
cation  of  automated  training  concepts  to  LSO 
training.  This  system  has  been  through  one 
implementation  and  testing  iteration  and  is 
currently  under  revision  based  on  initial 
test  results.  This  paper  describes  the 
implementation  and  testing  of  that  system. 

SYSTEM  DESIGN 

The  basis  for  design  of  a  laboratory  LSO 
training  system  was  an  LSO  training  system 
requirements  study  (Hooks,  Butler,  Gullen  and 
Petersen,  1978)  sponsored  by  NTEC.  This 
study  examined  the  LSO  job  and  the  existing 
LSO  training  program  to  identify  problems  and 
candidate  solutions.  An  automated  training 
system  concept  emerged  as  a  very  promising 
and  potentially  cost-effective  aid  to  LSO 
training.  The  implication  was  analogous  to 
synthetic  pilot  training:  an  instructional ly 
controlled  environment,  independent  of  costly 
carrier  operations,  for  promoting  the  acqui¬ 
sition  of  basic  waving  skills  and  for  exer¬ 
cise  of  these  skills  in  critical  operational 
situations,  without  jeopardizing  safety 
considerations.  Other  study  results  included 
the  identification  of  LSO  tasks  and  learning 
objectives,  development  of  a  syllabus  for 
skill  acquisition,  delineation  of  candidate 
functions  for  an  automated  LSO  training  sys¬ 
tem  and  an  examination  of  the  status  of 
training  system  technologies  which  were  rele¬ 
vant  to  the  candidate  functions. 

The  functional  architecture  of  an  auto¬ 
mated  LSO  training  system  supported  the 
identification  of  key  system  feasibility 
issues.  One  overall  issue  concerned  the 
feasibility  of  automating  LSO  task  interac¬ 
tion  through  the  integration  of  graphics. 


aircraft  dynamics  modelling  and  automated 
speech  recognition.  This  issue  appeared 
very  appropriate  to  investigation  in  the 
laboratory  environment.  Another  issue  appro¬ 
priate  to  laboratory  investigation  was  the 
feasibility  of  automated  LSO  performance 
measurement.  The  relevance  of  this  issue 
was  enhanced  by  the  fact  that  another  NTEC 
sponsored  activity  was  underway  to  define 
the  behavioral  aspects  of  the  LSO  waving  task 
(Borden  and  McCauley,  1978).  Development  of 
a  laboratory  LSO  training  system  also  had  the 
potential  to  support  investigations  of 
instructional  features  as  well  as  alternative 
strategies  and  techniques  for  promoting  the 
effective  acquisition  of  LSO  decision-making 
skills.  These  considerations  resulted  in  the 
initial  design  of  a  laboratory  LSO  training 
system  to  be  implemented  in  computer  facili¬ 
ties  at  the  Human  Factors  Laboratory  of  NTEC. 

The  primary  elements  of  the  laboratory 
LSO  training  system  include  graphic  display 
of  the  carrier  approach  scene,  a  model  of 
pilot  and  aircraft  approach  dynamics,  auto¬ 
mated  speech  recognition,  behavior  model 
based  performance  evaluation,  scenario  based 
exercise  control  and  exercise  data  recording. 
The  primary  hardware  components  employed 
include  a  Megatek  5000  random  scan  display, 
a  NOVA  1200  CPU,  a  NOVA  800  CPU  with  floating 
point  and  a  VIP  100  voice  recognition 
preprocessor. 

The  detailed  design  was  a  joint  effort 
between  Logicon  and  NTEC  personnel.  Logicon 
designed  the  software  for  the  NOVA  800  CPU 
which  included  the  graphics,  exercise  con¬ 
trol,  performance  evaluation  and  data  record¬ 
ing  elements.  NTEC  personnel  were  responsi¬ 
ble  for  the  pilot/aircraft  model  and  speech 
recognition  elements  which  were  to  be  per¬ 
formed  in  the  NOVA  1200  CPU.  The  design 
effort  involved  frequent  interaction  among 
training  analysis,  LSO  subject  matter  expert 
and  programming  personnel.  Training  analysis 
and  subject  matter  expert  inputs  to  the 
design  included: 

-  parameters  for  scenario  control 

-  LSO  vocabulary  requirements 

-  instructional  prompting  and  aiding 

-  exercise  data  recording  requirements 

-  performance  evaluation  criteria 

-  "pilot"  control  variations 

-  instructional  scenario  definition 

-  LSO/pilot  interaction  contingencies. 


SYSTEM  IMPLEMENTATION 
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The  operating  concept  for  the  training 
system  was  to  allow  the  subject  to  view  a 
graphic  portrayal  of  the  approach,  perceive 
the  need  for  a  voice  call  and  input  a  voice 
call  through  the  speech  recognition  element 
of  the  system.  The  system  would  then  process 
this  call,  simulate  and  display  the  pilot 
response  and  record  an  evaluation  of  the  sub¬ 
ject's  performance.  Thus  the  system  would 
provide  closed-loop  LSO  and  pilot  interaction 
in  an  instructional  scenario  controlled 
envi  ronment. 

The  hardware  available  for  system  imple¬ 
mentation  was  limited  in  capacity  for  han¬ 
dling  the  desired  functional  capabilities. 
Therefore  several  design  simplifications  were 
required.  For  control  of  aircraft  dynamics, 
the  approach  was  segmented  into  a  limited 
number  of  "zones"  defining  variations  in 
glideslope,  lineup  and  angle  of  attack  (AOA) 
which  are  oriented  about  the  optimum  param¬ 
eters  for  these  dimensions.  The  closing 
approach  speed  of  the  aircraft  was  fixed  at 
110  knots.  Fixed  variations  in  sink  rate 
and  drift  rate  were  also  established.  These 
simplifications  allow  fairly  comprehensive 
scenario  control  using  minimal  CPU  resources. 

The  performance  evaluation  function  was 
also  simplified.  The  behavioral  model  devel¬ 
oped  under  another  NTEC  sponsored  effort 
(Borden  and  McCauley,  1978)  was  too  complex 
for  available  CPU  resources.  Therefore,  an 
evaluation  model  was  devised  which  was  based 
on  the  correctness  and  timeliness  of  ten 
basic  LSO  phrases  and  the  general  criteria 
for  their  relevance.  To  minimize  complexity, 
the  aircraft  state  parameters  used  to  estab¬ 
lish  these  criteria  only  included  glideslope 
and  lineup  position,  AOA,  and  range  segment 
(start,  in  the  middle,  in  close,  at  the 
ramp).  It  is  essentially  a  "snapshot"  model 
since  rates  of  aircraft  parameter  changes 
(sink  and  drift  rate)  are  not  accountable. 
However,  the  model  implementation  is  such 
that  it  can  be  increased  in  complexity  if 
additional  CPU  resources  become  available 
(as  in  the  case  of  implementing  the  system 
in  higher  capacity  computer  hardware). 

Another  limitation  of  this  element  is  that 
it  does  not  look  for  errors  of  omission 
ii  .g.  no  call  made  when  one  is  required). 
Although  simplified,  this  mechanization  still 
provides  an  adequate  representation  of  an 
automated  performance  evaluation  capability 
for  LSO  training  research. 

The  aircraft  image  displayed  has  minimal 
fidelity,  looking  somewhat  like  a  paper  air¬ 
plane.  This  significantly  minimizes  graphics 
processing  requirements. 


The  results  of  LSO/pilot  interaction 
were  also  simplified.  A  correct  call  by  the 
LSO  results  in  a  maneuver  by  the  pilot  to  put 
the  aircraft  in  an  optimum  state  (correct 
glideslope,  lineup,  AOA).  An  incorrect  call 
by  the  LSO  is  "ignored"  by  the  pilot. 

To  provide  supplemental  perceptual  aid¬ 
ing  to  the  subject,  an  aircraft  positioning 
grid  was  incorporated  into  the  graphic  scene. 
This  grid  provides  a  graphic  indication  of 
aircraft  position  relative  to  the  glideslope 
and  lineup  "zones"  and  range  segments.  Dis¬ 
play  of  this  grid  during  an  approach  is  a 
pre-selectable  option  for  the  system  operator 
(instructor).  Another  pre-selection  option 
that  was  incorporated,  is  a  text  message  that 
provides  scenario  generated  prompting  and 
real-time  performance  evaluation  feedback. 

The  real-time  feedback  text  displays  the 
phrase  recognized  by  the  system  and  the  cor¬ 
rect  phrase  (if  different)  as  determined  by 
performance  evaluation. 

Scenarios  implemented  into  the  system 
are  preprogrammed  approach  profiles  which  are 
keyed  to  the  instruction  of  the  decision¬ 
making  situations  imbedded  in  the  performance 
evaluation  model.  Scenario  commands  provide 
for  aircraft  maneuvers  within  the  approach 
segmentation  concept  described  earlier.  They 
also  provide  pre-determi ned  textual  prompting 
of  voice  calls  appropriate  to  the  decision 
situation  presented. 

From  an  overall  viewpoint,  the  labora¬ 
tory  LSO  training  system  is  a  limited  repre¬ 
sentation  of  all  the  major  functional  ele¬ 
ments  of  an  automated  LSO  training  system, 
with  the  exception  of  automated,  adaptive 
syllabus  control . 

SYSTEM  TESTING 

After  implementation,  the  system  was 
demonstrated  to  fleet  LSOs  and  an  informal 
test  was  conducted  in-house  at  Logicon.  Two 
subjects  were  used  in  the  test,  each  having 
a  Navy  carrier  pilot  background,  but  no  LSO 
experience.  Over  two  days,  each  subject  was 
sequenced  through  exercises  representative  of 
a  syllabus  based  on  incremental  acquisition 
of  waving  skill  components  (glideslope,  line¬ 
up,  AOA  and  waveoff  related  calls).  From  the 
standpoint  of  system  operability,  the  system 
tested  successfully.  From  a  training  stand¬ 
point,  however,  several  deficiencies  were 
di  scovered. 

One  of  the  major  discrepancies  involved 
difficulties  on  the  perception  of  approach 
deviations  presented  by  the  scenarios.  This 
led  to  a  conclusion  that  some  amount  of 
parameter  exaggeration  was  required  to  enable 
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meaningful  presentation  of  waving 
situations. 


Absence  of  a  capability  to  detect  errors 
of  omission  significantly  lessened  the  value 
of  the  performance  evaluation  function. 

It  became  evident  during  testing  that  a 
freeze  capability  was  necessary  for  effective 
conduct  of  a  training  session. 

The  aircraft  position  grid  proved  to  be 
excessively  cluttered,  thus  lessening  its 
effectiveness  as  a  perception  aid. 

The  fact  that  the  aircraft  responded 
only  to  correct  ISO  calls  turned  out  to  be 
an  excessive  departure  from  real-world  LSO 
waving  interaction. 

There  were  speech  recognition  difficul¬ 
ties  which  were  attributed  to  lack  of  a 
contextual  voice  data  collection  effort. 

This  capability  was  implemented  at  NTEC  but 
was  not  used  in  Logicon's  in-house  testing. 

It  was  noted  that  the  performance  eval¬ 
uation  feedback  feature  would  have  been  more 
effective  if  clarifying  information  could 
have  accompanied  the  display  of  the  correct 
call. 

Design  of  the  glideslope  and  lineup 
"zones"  to  converge  at  the  optimum  touchdown 
position  significantly  degraded  the  depiction 
of  deviations  for  the  "in  close"  and  "at  the 
ramp"  segments  of  the  approach.  It  also  pre¬ 
cluded  allowing  the  results  of  the  approach 
*o  vary.  The  system  was  thus  unable  to  pro¬ 
vide  additional  feedback  on  the  effectiveness 
of  LSO  control  in  the  terminal  portion  of 
the  approach. 

There  was  some  question  regarding  the 
fidelity  of  the  aircraft  image  from  a  train¬ 
ing  standpoint. 

It  was  also  noted  that  some  form  of 
evaluative  feedback  should  be  presented  to 
the  subject  by  the  system  following  the 
approach  in  order  to  enhance  learning  rate. 

Another  result  of  testing  and  demonstra¬ 
tion  of  the  system  to  skilled  LSOs  was  the 
realization  that  the  laboratory  system,  with 
refinement,  has  significant  potential  as  an 
introductory,  part-task  LSO  decision  training 
system.  It  could  provide  limited  "hands  on" 
training  for  a  naive  trainee  in  an  instruc¬ 
tional  setting  such  as  the  LSO  Phase  1  School. 

SYSTEM  REVISION 

In  response  to  the  demonstration  and 
testing  results  described  above,  the  labora¬ 
tory  LSO  training  system  is  currently 


undergoing  revision.  Several  objectives 
have  been  established  to  guide  software  revi¬ 
sion  and  research  utilization  of  the  revised 
system. 

The  first  objective  is  to  enhance  the 
system  in  response  to  the  "lessons  learned" 
earlier  in  the  project.  One  orientation  to 
the  enhancement  is  to  improve  the  perceptual 
characteristics  of  the  instructional  presen¬ 
tation.  This  is  necessary  to  minimize  the 
instructional  emphasis  on  learning  the  per¬ 
ceptual  aspects  of  the  LSO  decision  task  and 
to  enhance  LSO  acceptance  of  the  system  as  a 
potential  training  system  concept.  The  other 
orientation  is  toward  improved  instructional 
features  to  support  LSO  decision  learning. 

A  second  objective  is  to  evaluate  the 
effectiveness  of  providing  "hands  on" 
decision-oriented  training  in  the  early  stage 
of  the  LSO  learning  process.  The  current  LSO 
training  philosophy  employs  a  passive,  obser¬ 
vational  strategy  which  focuses  on  the  per¬ 
ceptual  aspects  of  the  LSO  tasks.  Early 
"hands  on"  instruction  should  promote  a 
clearer  understanding  of  the  relationships 
between  task  stimuli  and  the  LSO  decision 
process  and  should  enable  the  trainee  to  be 
more  intellectually  active  in  the  observa¬ 
tional  stage  of  training  aboard  ship.  These 
factors  have  significant  potential  for  accel¬ 
erating  skill  acquisition  in  Phase  III  train¬ 
ing  aboard  ship. 

A  third  objective  is  to  evaluate  the 
feasibility  of  providing  some  limited  levels 
of  instructorless  training  to  LSOs.  The 
factors  in  this  feasibility  investig  tion  are 
training  effectiveness  and  user  (LSO)  accept¬ 
ance.  Instructorless  training,  even  for 
small  portions  of  LSO  training,  can  provide 
significant  benefit  in  view  of  existing  and 
forecasted  instructor  LSO  shortages. 

Although  the  system  will  not  be  totally 
instructorless,  the  automated  performance 
feedback  features  will  support  investigation 
of  instructorless  training  concepts.  In  con¬ 
junction  with  this  objective,  it  is  antici¬ 
pated  that  some  beneficial  knowledge  will  be 
gained  concerning  effective  methods  and  con¬ 
tent  of  performance  feedback  for  an  LSO 
training  system. 

The  final  objective  planned  for  this 
effort  involves  evaluating  the  utility  of  a 
simplified  LSO  decision-making  model  to  early 
LSO  training.  The  LSO  decision-making 
process  is  extremely  complex  and  difficult 
to  verbalize.  These  factors  are  largely 
responsible  for  the  lengthy  LSO  training 
process.  Over  time,  the  trainee,  through 
extensive  observation  of  skilled  LSOs, 
acquires  knowledge  of  the  decision  process. 
Providing  the  trainee  with  simplified,  but 
increasingly  complex,  models  of  the  decision 
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process  should  promote  more  efficient  acqui¬ 
sition  of  LSO  decision-making  skills.  This 
system  provides  a  simplified  LSO  decision 
model  as  the  basis  of  its  performance  eval¬ 
uation  capability. 

There  is  also  an  underlying,  secondary 
objective  within  this  project.  Since  the 
system  itself  has  potential  as  a  cost- 
effective  automated  "part  task"  training 
system  for  the  initial  stage  of  LSO  training, 
this  possibility  will  be  assessed  informally 
during  experimental  activities. 

To  meet  these  objectives  several  func¬ 
tional  revisions  are  currently  planned. 

The  revisions  are  delineated  below: 

Freeze 

-  operator  initiated  in  run-time 

-  also,  possibly  “freeze  on  error" 

Feedback  Messages 

-  in  expanded  text  area 

-  information  on  call  recognized,  cor¬ 
rect  call,  relevant  approach 
paramete  rs 

-  available  during  approach 

-  selectable  during  freeze 

Detection  of  "Missed  Calls"  by  Perform¬ 
ance  Evaluation 

Graphics  Improvements 

-  "ladder  line"  to  be  added  to  deck 
scene 

-  display  of  improved  aircraft  image 
during  freeze 

-  improved  perception  aid  similar  to  LSO 
HUD 

End  of  Run  Feedback  Data  on  Megatek 

-  approach  result  (wire,  lineup) 

Aircraft  Response  to  Incorrect  Calls 

-  response  to  incorrect  calls  which  are 
considered  imperative  (power,  waveoff, 
right/left  for  lineup) 

End  of  Run  Data  to  Operator  CRT  and  Stu¬ 
dent  Records 

-  "snapshot"  data  for  each  call  and 
error  occurrence  during  approach 

-  approach  results 

Revision  of  Glideslope  and  Lineup  Zone 

Origin 

-  to  enable  discrimination  in  approach 
parameters  on  touchdown  (wi res ,  1  ineup) 


Exaggeration  of  Lineup,  Glideslope  and 

AOA  Deviations 

-  to  enhance  student  perception  of 
devi  ations 

Data-Driven  Prompting  During  Approach 

-  same  fext  area  and  format  as  feedback 
messages 

-  scenario  driven  prompting  to  be 
deleted 


The  design  and  implementation  of  these  revi¬ 
sions  is  currently  in  progress.  After  the 
completion  of  this  revision  process,  a  pilot 
study  will  be  conducted  at  Logicon  to  verify 
system  operability  and  to  informally  address 
the  issues  described  in  the  objectives  pre¬ 
sented  earlier.  The  pilot  study  will  also 
support  the  design  of  formal  experimentation 
to  be  conducted  at  the  NTEC  Human  Factors 
Laboratory. 


SUMMARY 

The  critical  shortage  of  skilled  LSOs 
continues  to  plague  the  Navy.  This  research 
effort,  employing  a  laboratory  LSO  training 
system,  is  exploring  means  for  accelerating 
the  acquisition  of  LSO  skills.  The  early 
indications  from  this  and  other  NTEC- 
sponsored  efforts  are  encouraging,  thanks  to 
the  cooperative  interaction  among  government 
and  industry  researchers  and  fleet  LSOs. 
Advanced  training  technologies  appear  to 
hold  great  promise  for  he  ing  solve  the 
LSO  training  problem. 
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INSTRUCTOR  MODEL  CHARACTERISTICS  FOR  THE  LSO  - 
THE  BRIDGE  BETWEEN  6.1  and  6.2 


DR.  DOUGLAS  C.  CHATFIELD 
Behavioral  Evaluation  &  Training  Systems 


INTRODUCTION 

The  present  paper  is  an  adaptation 
of  a  somewhat  larger  report  entitled 
"Instructor  Model  Characteristics 
for  Automated  Speech  Technology"  by 
Chatfield,  Marshall,  and  Gidcumb 
which  will  be  available  soon.  In 
this  larger  report,  the  task  require¬ 
ments  of  the  Landing  Signal  Officer 
(LSO)  were  reviewed  along  with  those 
of  the  air-intercept  controller  (AIC) 
and  the  ground-controlled  approach 
controller  (GCA)  in  terms  of  recent 
developments  in  the  6.1  or  basic  re¬ 
search  literature.  The  present  paper 
is  an  attempt  to  review  some  of  the 
more  important  concepts  which  could 
lead  to  an  instructor  model  for  the 
development  of  LSO  training  systems. 

At  the  outset,  it  would  be  good 
to  establish  what  we  have  in  mind 
when  we  refer  to  an  instructor  model. 
Imagine  for  a  moment  the  character¬ 
istics  of  a  human  instructor: 

1 )  He  possesses  a  knowledge  base 
which  he  ultimately  transmits  to  the 
students, 

2)  He  attends  to  various  perfor¬ 
mance  indeces  to  gauge  student  pro¬ 
gress, 

3)  He  possesses  a  set  of  training 
techniques  which  he  invokes  in  the 
training  process, 

4)  He  possesses  an  ability  to  self 
evaluate  and  modify  these  training 
techniques  as  1  e  gains  experience 
with  increasing  numbers  of  students, 

5)  He  possesses  a  high  degree  of 
flexibility  in  that  he  can  make  real¬ 
time  decisions  regarding  instruc¬ 
tional  alternatives  throughout  the 
training  session.  He  is  capable  of 
using  individual  student  performance 
indeces  as  feedback  in  a  closed-loop 
system.  With  these  characteristics 
we  would  then  label  the  human  in¬ 
structor  as  an  intelligent  training 
system. 

Current  trends  in  research  on 
automated  instruction  are  moving  in 
the  direction  of  what  is  called 
"intelligent  knowledge-based  training 
systems."  In  other  words,  training 
systems  are  beginning  to  emulate  and 


model  the  flexibility  of  the  human 
instructor.  This  is  in  contrast  to 
the  more  rigid  "script-based"  sys¬ 
tems  in  which  training  exercises, 
frames,  etc.,  have  been  preprogrammed 
and  the  control  logic  of  the  system 
is  restricted  to  the  control  of 
essentially  a  branching  process. 

What  we  seek  is  a  control  logic 
in  a  training  system  which  can  emu¬ 
late  as  many  characteristics  of  the 
human  instructor  as  possible.  This 
we  would  refer  to  as  the  instructor 
model.  It  would  possess:  1)  a 
knowledge  or  skills  representation 
system  from  which  instructional  and 
training  exercises  could  be  generated 

2)  a  model  of  the  student  which  would 
describe  the  processes  by  which  a 
student  learns  and  would  summarize  a 
student's  current  learning  state, 

3)  a  generative  logic  which  could 
generate  instructional  scenarios  and 
exercises  that  would  optimize  margin¬ 
al  increments  in  learning  given  a 
student's  current  learning  state,  and 

4)  processes  by  which  the  control 
logic  might  modify  itself  to  increase 
its  efficiency  as  it  gains  experience 
with  increasing  numbers  of  students. 
It  is  these  4  characteristics  of  the 
instructor  model  which  we  will 
briefly  attempt  to  address. 

CURRENT  THEORY  ON  COMPLEX  COGNITIVE 
TA3KS. 

Before  looking  at  a  knowledge 
representation  system,  we  need  to 
characterize  the  skills  required  by 
the  LSO.  Similar  to  other  speech- 
based  training  tasks,  the  LGO's  job 
is  a  series  of  complex  event-driven 
cognitive  tasks.  The  LSO  must  pro¬ 
cess  complex  visual  and  auditory  in¬ 
puts,  make  real-time  decisions,  and 
produce  complex  verbal  responses, 
lie  must  be  able  to  encode  subtle 
visual  cues  and  process  heavy  infor¬ 
mation  loads  under  pressure  and  at 
a  rate  dictated  by  external  events. 

A  major  difference  between  the 
expert  and  the  novice  is  that  the 
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expert  seem6  to  be  able  to  simulta¬ 
neously  handle  multiple  inputs,  hold 
information  in  memory,  process  the 
information,  and  select  the  next  re¬ 
sponse  concurrent  with  the  execution 
of  the  present  verbal  response*  The 
novice,  on  the  other  hand,  seems  to 
be  analogous  to  the  person  who  can't 
walk  and  chew  gum  at  the  same  time. 

The  novice  seems  to  be  held  back  by 
a  limited  central  processing  capacity. 

The  basic  research  literature  has 
proposed  several  resource  competition 
models  based  on  the  assumption  of  a 
limited  central  processing  capacity. 
One  view  (Kahneman,  1973)  describes 
these  limited  resources  as  a  pool  of 
"effort"  which  may  be  allocated  among 
tasks.  To  increase  performance  on  a 
particular  task,  an  individual  would 
need  to  increases  his  allocation  of 
cognitive  resources  to  that  task. 

If  he  was  engaged  in  a  second  con¬ 
current  task  at  the  time,  and  was  al¬ 
ready  using  all  of  his  resources,  then 
he  must  necessarily  borrow  some  re¬ 
sources  from  the  second  task  to  invest 
in  the  first.  Thus  to  produce  an  in¬ 
crease  in  performance  on  one  task, 
performance  on  a  second  task  may  have 
to  suffer. 

Norman  and  Bo brow  (1975)  have 
proposed  a  resource  competition  model 
which  could  be  incorporated  into  the 
control  logic  of  our  instructor  model. 
In  their  view,  whenever  the  perfor¬ 
mance  on  a  task  can  be  increased  with 
an  increased  allocation  of  resources 
(e.g.  processing  effort  or  concentra¬ 
tion)  then  the  task  was  said  to  be 
resource-limited.  Whenever  the  perfor¬ 
mance  level  remains  invariant  to  in¬ 
creased  allocations  of  processing 
effort  (resources) f  the  task  is  said 
to  be  data-limited.  Here  the  perfor¬ 
mance  is  limited  by  either  poor  sig¬ 
nal  quality  (e.g.  low  signal-to-noise 
ratio)  which  was  referred  to  as 
signal  data  limits  or  an  inadequately 
stored  memory  representation  referred 
to  as  memory  data  limits. 

Norman  and  Bobrow  describe  what 
they  refer  to  as  a  resource  allocation 
function  in  which  task  performance  is 
related  to  a  single  hypothetical 
dimension  of  resource  allocation. 
Figure  1  shows  resource  allocation 
functions  for  two  tasks.  As  addi¬ 
tional  resources  are  allocated  moving 
from  point  A  to  point  B  on  the 
abscissa,  performance  increases  on 
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Figure  1 .  Two  example  resource 
allocation  functions. 


both  tasks  and  are  resource  limited 
at  all  points  in  that  range.  But 
as  the  allocation  is  further  in¬ 
creased  from  points  £  to  £,  no 
further  increase  in  task  I  is  pro¬ 
duced,  thus  task  I  is  now  data- 
limited  while  task  II  is  still  re¬ 
source  limited. 

Resource  allocation  functions 
are  largely  hypothetical,  but  the 
performance  of  one  task  can  and  has 
been  empirically  related  to  the  per¬ 
formance  of  another  task.  The  left 
panel  of  Figure  2  shows  such  a  func¬ 
tion  between  two  tasks  and  is 


Performance  Task  I  Resource  Allocation 


Figure  2.  Resource  allocation 
functions  and  the 
corresponding  POC. 


referred  to  as  a  Performance  Opera¬ 
ting  Characteristic  (POC).  The 
corresponding  resource  allocation 
function  in  the  right-most  panel 
shov/s  that  the  region  of  investi¬ 
gation  on  the  abscissa  is  limited 
to  the  i  to  £  region  referring  back 
to  Figure  1 .  As  can  be  seen,  as 
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resources  are  taken  from  one  task 
and  applied  to  the  other,  the  per¬ 
formance  on  the  first  tack  drops 
while  the  performance  on  the  second 
increases.  This  would  be  a  typical 
POC  wherein  both  tasks  are  resource 
limited. 

One  may  ask — How  then  may  we  ac¬ 
count  for  the  fact  that  the  expert 
controller  seems  quite  facile  with 
concurrent  inputs  and  responses? 

The  expert  describes  his  responses 
as  being  quite  automatic  in  nature, 
requiring  little  or  no  conscious 
processing  effort,  Norman  and 
3obrow  explain  that  with  practice, 
the  students  may  learn  to  become 
more  efficient  in  their  processing, 
maybe  by  eliminating  processing  steps 
or  learning  to  process  the  minimum 
relevant  data,  etc.  Thus  with 
learning,  the  student  approaches  his 
data-limited  asymptote  more  rapidly 
as  he  increases  his  resource  alloca¬ 
tions.  This  is  depicted  in  Figure  3* 


Figure  3»  Changes  in  the  resource 
allocation  functions  in 
various  stages  of 
training  A  through  D. 


With  a  resource  function  as  is  shown 
in  function  D,  the  expert  performing 
this  task  would  seem  to  be  able  to 
do  so  independently  of  other  tasks 
in  that  with  very  little  processing 
effort,  he  achieves  data-limitation 
status.  Thus  as  described  here,  the 
performance  on  this  task  may  not 
really  be  "automatic"  but  simply 
"cheap"  in  terms  of  the  amount  of 
resources  required  to  reach  asymptote. 

The  concept  of  task  performance 
being  a  function  of  resource  alloca¬ 
tion  can  be  extended  to  the  task  of 
learning  itself.  All  of  us  have  ob¬ 
served  a  human  instructor  training  a 


student  in  a  complex  tack  telling 
the  student  to  "concentrate"  on 
certain  parts  of  the  task  at  various 
points  in  training.  The  assumption 
implied  in  making  the  student  con¬ 
centrate  on  one  part  of  the  tack  at 
the  expense  of  others  is  that  learn¬ 
ing  rate  is  positively  related  to 
the  amount  of  resources  invested 
(e.g.  concentration).  This  assump¬ 
tion  would  then  suggest  the  concept 
of  a  resource  allocation-acauisition 
function  as  shown  in  Figure  4.  Ac 


Figure  4.  Exemplar  resource 

allocation-acquisition 

function. 


can  be  seen,  the  momentary  learning 
rate  is  portrayed  hypothetically  as 
a  function  of  the  amount  of  resources 
allocated.  In  this  view,  the  rate 
at  which  a  task  is  mastered  could  be 
altered  by  manipulating  the  resources 
allocated. 

Thus  in  the  training  of  an  LSO, 
a  complex  aggregation  of  seemingly 
concurrent  tasks,  the  control  logic 
of  the  system  should  be  able  to  make 
real-time  adjustments  and  instruc¬ 
tional  decisions  regarding  the  de¬ 
mands  of  the  task,  such  that  the 
novice  possesses  enough  surplus  re¬ 
sources  to  invest  in  the  "learning 
of  a  new  part  of  the  task."  The  de¬ 
tails  of  how  this  could  be  done  are 
described  more  fully  in  Chatfield, 
et.al.  (1979)  and  Norman  (1978). 
Briefly  what  is  being  described  is 
the  adaptive  training  function  of 
the  system. 

Consider  the  complex  aggregation 
of  tasks  such  as  required  of  the  LSO 
as  being  decomposed  into  various  sub¬ 
tasks  and  parts  for  training  purposes. 


The  adaptive  functions  of  the  system 
would  then  control  the  subtask  presen¬ 
tation  schedule.  As  a  student  is 
working  on  a  particular  portion  of  the 
task,  the  system  would  not  add  new 
subtaskc  to  the  composite  until  the 
current  portion  of  the  task  became 
resource  efficient,  i.e.,  the  resource 
allocation  function  were  surmised  to 
look  something  like  function  D  in 
Figure  3.  If  the  new  subtask  were  to 
be  added  too  early,  and  the  resource 
allocation  function  of  the  current 
composite  of  tasks  was  that  of  A  in 
Figure  3,  then  the  addition  of  the 
new  subtask  would  have  one  of  two 
effects.  If  the  student  invested 
some  of  his  limited  resources  in  the 
new  subtask,  his  current  performance 
level  on  the  current  tasks  would  fall. 
If  on  the  other  hand,  he  chose  to 
maintain  his  current  performance 
levels,  he  may  have  to  virtually  ig¬ 
nore  the  demands  of  the  new  subtask 
until  the  tasks  on  which  he  is  cur¬ 
rently  working  become  more  "automatic" 
thereby  releasing  precious  resources. 
The  actual  technology  of  how  an  in¬ 
telligent  system  might  solve  for  the 
"optimal"  point  in  training  at  which 
a  now  subtask  might  be  scheduled  can¬ 
not  be  discussed  here,  but  some  may 
want  to  refer  to  Chat  field  and  Gidcumb 
(1977)  or  Chant  and  Atkinson  (1973) 
for  the  mathematics  and  theory  invol¬ 
ved. 

In  addition  to  the  example  on  task 
scheuuling,  there  are  several  other 
ways  for  the  adaptive  function  of  the 
system  to  adjust  the  demands  of  the 
tasks  during  training.  The  main  point 
to  be  discussed  here  however,  is  the 
idea  that  the  adaptive  logic  in  the 
system  should  be  based  on  the  theory 
on  resource  limitations  currently 
available  in  the  6.1  and  basic  re¬ 
search  literature. 

In  addition  to  an  adaptive  func¬ 
tion,  the  instructor  model  of  an  in¬ 
telligent  system  would  incorporate 
a  tutorial  function.  By  the  tutorial 
function,  we  mean  the  establishment 
of  knov/ledge  structures,  the  imparting 
of  facts  or  rules  of  behavior  by  the 
system.  As  an  example,  say  that  a 
student  makes  an  incorrect  advisory 
on  an  approach.  The  error  may  have 
two  different  causes.  One  is  that 
the  student  may  have  known  what  the 
correct  advisory  should  have  been, 
but  got  confused  and  committed  the 
error  when  "two  many  things  happened 


at  once"  or  in  other  words,  proces¬ 
sing  demands  were  too  great,  here 
we  would  say  that  the  student  w as 
resource  limited.  Presumably,  if 
the  approach  were  repeated  with 
fewer  concurrent  events  demanding 
some  of  his  resources,  he  would  be 
able  to  devote  more  of  nis  resources 
to  the  advisory  in  question  and  would 
not  make  the  error.  However,  if  the 
demands  of  the  task  were  decreased 
and  the  student  still  made  an  incor¬ 
rect  advisory,  then  we  would  infer 
that  the  error  was  due  to  a  second 
cause.  It  may  be  that  the  student 
had  forgotten  certain  rules  on  the 
generation  of  advisories,  or  maybe 
he  had  never  known  them.  Now  his 
performance  is  data  limited  or  what 
Iiorman  referred  to  as  memorv-data- 
limited.  The  idea  here  is  that  no 
matter  how  much  of  his  resources  the 
student  may  invest,  performance  will 
not  improve  with  increased  resource 
allocations.  Thus  the  system  would 
need  to  enter  its  tutorial  function. 
Here  the  system  might  interrupt  the 
continuous  approaches  of  simulated 
aircraft  and  simply  explain  the 
needed  rules  for  the  advisory  in 
question.  This  would  be  analogous 
to  a  human  instructor  stopping  the 
practice  session  for  a  bit  of  verbal 
instruction. 

The  idea  that  a  training  system 
would  alternate  between  tutorial  and 
adaptive  functions  rests  on  the 
assumption  of  a  process  vs.  structure 
distinction  currently  found  in  the 
literature.  Structure  refers  to 
units  of  knowledge  or  schemata, 
whereas  process  refers  to  the  acti¬ 
vation  of  the  schemata  or  the  utili¬ 
zation  of  the  units  of  knowledge  in 
processing  incoming  stimuli  for 
eventual  response  production.  Struc¬ 
ture  is  established  through  more 
didactic  types  of  instruction,  whereas 
process  becomes  more  automatic  or 
resource  efficient  through  practice. 
One  of  the  requirements  of  the  system 
would  be  the  capability  to  vary  task 
demands  in  diagnosing  errors  as  being 
caused  by  either  processing  or  struc¬ 
tural  limitations. 

KNOWLEDGE- BASED  GENERATIVE  SYSTEMS 

Recent  research  in  training  sys¬ 
tems  funded  by  the  Office  of  Naval 
Research  (ONR)  has  progressed  to  what 
is  referred  to  as  knowledge-based 
generative  systems  (see  Wescourt. 


et.al.,  1977)*  The  term  "generative” 
is  used  in  a  way  to  depict  the  sys¬ 
tems  capability  to  generate  its  own 
questions  and  exercises  for  the  stu¬ 
dents  rather  than  being  limited  to 
a  preprogrammed  series  of  CAI-style 
frames.  Further,  the  system  may 
even  be  able  to  support  student  ini¬ 
tiative,  i.e.,  questions  from  the 
students  wherein  unique  answers  are 
generated  by  the  system  (see 
Carbonell,  1970)*  To  be  able  to 
generate  and  formulate  questions  and 
answers,  the  system  must  possess  the 
same  knowledge-base  that  the  student 
is  to  acquire.  Currently  ONR  is 
funding  projects  investigating  the 
representation  of  knowledge,  and 
ultimately  how  the  representation 
might  be  used  in  an  automated  train¬ 
ing  system. 

Most  of  the  recent  work  can  be 
described  as  an  incorporation  of 
artificial  intelligence  in  automated 
instruction.  Much  of  the  work  how¬ 
ever,  has  centered  around  the  repre¬ 
sentation  of  structure  and  may  be 
somewhat  limited  in  the  representa¬ 
tion  of  process.  As  an  example, 
Goldstein  (1979)  presents  what  he 
calls  a  "genetic  graph"  to  represent 
knov/ledge.  The  idea  of  the  genetic 
graph  is  that  it  attempts  to  capture 
the  evolutionary  process  by  which  a 
mature  set  of  procedures  evolve.  In 
Figure  5  it  can  be  seen  that  the 
nodes  representing  the  rules  are 


gencralization/speci fi- 
cation  (G/G)  and 
analogy  (A). 


linked  by  relationships  of  evolution. 
In  the  example,  all  the  rules  shown 
evolve  from  rule  A  which  was  learned 
initially.  Rule  is  said  to  be  a 
refinement  of  rule  A  while  rule  ABC 
is  a  version  that  is  refined  still 
further.  Rules  ADD.  AdD,  and  adD 
can  be  viewed  as  more  general  (or 
more  specific)  versions  of  rule  AD. 
and  are  analogous  to  each  other. 

One  advantage  of  the  genetic  graph 
is  that  it  represents  the  student's 
knowledge  at  various  stages. 

A  second  advantage  is  that  it 
provides  the  system  with  a  means  by 
which  it  can  present  new  information 
in  terms  of  previously  learned  rules. 
Take  for  example  rule  ADD  in  Figure  % 
This  rule  could  be  presented  to  the 
student  as  a  generalization  of  rule 
AD.  as  being  analogous  to  rule  adD 
or  being  analogous  to  rule  AdD  pro¬ 
viding  that  these  other  rules  were 
already  known.  Thus  the  genetic 
graph  provides  the  .potential  for  a 
very  powerful  means  by  which  the  sys¬ 
tem  can  make  use  of  prior  knowledge 
in  generating  multiple  styles  by 
which  a  rule  could  be  presented. 

Thus  if  a  system  can  generate  tasks, 
questions  or  responses  to  student 
queries  from  an  appropriately  repre¬ 
sented  knov/ledge  domain,  it  can 
possess  the  flexibility  required  to 
provide  truly  individualized  instruc¬ 
tion  as  would  a  human  tutor. 

The  research  on  competition  for 
central  resources  however,  points 
out  an  important  need.  The  adaptive 
logic  of  an  automated  speech-based 
training  system  will  need  to  be  in¬ 
volved  in  resource  budgeting 
during  the  training  of  complex  con¬ 
current  tasks.  Let  us  assume  a 
simple  network  of  tasks  which  rep¬ 
resent  those  relationships  as  shov/n 
in  Figure  6.  Let  us  further  assume 
for  illustration  that  we  could 
quantify  task  demand  for  central  re¬ 
sources  as  a  percentage  figure,  both 
for  tasks  executed  in  isolation  and 
concurrently. 

As  can  be  seen  in  Figure  6,  tasks 
£  and  £  are  represented  as  independ¬ 
ent  tasks.  If  truly  independent, 
then  presumably  the  student  could  be 
required  to  perform  at  both  tasks 
without  both  of  them  competing  for 
the  same  central  resources.  Tasks 
A  and  B  however,  represent  what  we 
will  call  a  symbiotic  relationship 
wherein  the  concurrent  performance 
on  task  B  might  even  augment  the 


Figure  6.  Network  of  five  tasks 
with  the  links  repre¬ 
senting  Symbiotic  (S), 
Competitive  (C)  or 
Independent  (I)  pair¬ 
wise  relationships. 


performance  on  A  and  vice  versa. 

Tasks  D  and  C  represent  a  more  prob¬ 
able  type  of  relationship — a  competi¬ 
tive  relationship.  If  both  demand 
70%  of  the  available  resources  sepa¬ 
rately,  then  their  competitive  rela¬ 
tionship  would  imply  that  their  joint 
performance  might  demand  more  than 
the  simple  sum  of  their  separate  de¬ 
mands — 160%,  Since  the  student  can 
give  only  100%  in  total,  one  or  the 
other  or  both  of  the  tv/o  tasks  must 
suffer  with  the  student  in  this  over¬ 
load  situation. 

The  adaptive  logic  of  the  instruc¬ 
tor  model  could  do  one  of  three 
things  at  this  point.  First,  the 
student  could  be  trained  in  isolation 
on  task  C  until  it  reaches  automati- 
city  or  becomes  more  resource  effi¬ 
cient.  Then  when  10%  of  the  re¬ 
sources  in  isolation  produce  the 
same  required  level  of  performance 
ao  a  70%  investment  did,  task  D 
could  then  be  introduced.  Because 
of  the  advance  training  on  C,  the 
joint  demand  of  the  two  tasks  is 
now  manageable. 

A  second  possibility  for  the  adap¬ 
tive  logic  of  the  instructor  model 
would  be  to  reduce  the  joint  demand 
of  the  two  tasks.  This  could  be 
done  by  adjusting  the  adaptive  vari¬ 
ables  on  one  or  both  of  the  tasks 
(e.g.  the  pace  could  be  slowed  by 
adjusting  the  airspeed  of  the 
approaching  aircraft).  This  would 
be  a  more  likely  instructional 
strategy  to  take  if  the  training  of 
tasks  in  isolation  presents  a  real 
compromise  in  simulation  fidelity. 


It  may  be  that  both  previous 
approaches  represent  undesireable 
compromises,  and  that  the  overload 
situation  simply  cannot  be  avoided. 
The  joint  performance  will  simply 
have  to  suffer  until  practice 
eventually  makes  the  tasks  more  re¬ 
source  efficient.  In  that  case, 
the  adaptive  logic  takes  on  the 
duties  of  resource  budgeting.  By 
telling  the  student  to  attend  to, 
or  concentrate  on,  one  task  more 
than  the  other,  it  would  increase 
the  rate  in  which  that  task  would 
be  mastered  (become  resource  effi¬ 
cient).  As  discussed  previously, 
the  job  of  the  instructor  model 
would  be  to  solve  for  the  optimal 
allocation  of  resources  during  the 
joint  training  of  £  and  D.  Thus  in 
order  for  the  intelligence  of  the 
system  to  possess  the  flexibility 
to  generate  event-driven  exercises 
such  as  the  simulated  aircraft  ap¬ 
proaches,  the  instructor  model 
would  need  to  incorporate  a  net¬ 
work  representation  of  pairwise 
relationships  regarding  resource 
requirements  between  tasks. 

MODEL  OF  THE  STUDENT 

In  addition  to  the  representa¬ 
tion  of  knowledge,  all  the  training 
systems  described  previously  have 
some  method  for  representing  the 
student  in  his  progress  through  the 
course.  One  form  of  the  representa¬ 
tion  is  to  view  the  student  as  a 
grov/ing  subset  of  skills  or  know¬ 
ledge  units.  The  system  keeps  a 
record  of  which  skills  have  been 
mastered  and  which  are  left  to  be 
acquired.  In  its  simplest  form 
then,  each  skill  or  knowledge  struc¬ 
ture  can  occupy  one  of  only  tv/o 
states:  learned  or  unlearned. 

Goldstein  (1979)  in  his  genetic 
graph,  provided  a  different  tech¬ 
nique  for  representing  the  progress 
of  a  skill.  Instead  of  representing 
the  skill  at  a  single  location  in 
the  network  wherein  the  skill  occu¬ 
pied  one  of  two  learning  states,  he 
represented  the  skill  at  several 
locations  in  the  graph  wherein  the 
various  locations  represented  evol¬ 
utionary  stages,  ilis  reasoning  was 
that  it  is  not  really  the  same  skill 
at  those  various  locations  but 
rather  qualitatively  different  and 
increasingly  refined  versions  of 
the  skill. 
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./hat  we  would  require  in  our  ideal 
system  is  possibly  both  forms  of  rep¬ 
resentation:  first,  a  single  loca¬ 

tion  multi-learning  state  represen¬ 
tation  for  structural  units  that  do 
not  undergo  change  as  they  are 
learned  (see  Atkinson,  1976);  and 
secondly,  a  multi-location  repre¬ 
sentation,  such  as  in  the  genetic 
graph,  when  the  skill  or  structural 
unit  undergoes  some  transformation 
or  refinement  as  learning  progresses. 
.Ve  would  also  desire  thirdly,  a  con¬ 
tinuous  representation  of  the 
strength  of  a  skill  or  schema.  This 
last  requirement  would  yield  informa¬ 
tion  regarding  the  degree  of  auto- 
maticity  of  a  skill  which  should  be 
represented.  Finally,  we  would  also 
desire  to  the  extent  possible,  that 
the  representation  of  the  progress 
of  a  structural  unit  be  formalized 
in  stochastic  form,  which  would 
allow  the  adaptive  logic  in  the 
system  to  optimize  its  instructional 
decisions  (see  Chatfield  and  Gidcumb, 
1977). 

o  JLF  MODI  FI  CATI Oil  PROPERTI  .13 

An  additional  and  desireable 
property  of  on  intelligent  system 
is  that  it  have  the  capacity  for 
self  modification.  As  the  instructor 
model  obtains  additional  data  from 
the  students,  it  would  be  able  to 
benefit  from  that  experience  and  be¬ 
come  a  more  efficient  training  sys¬ 
tem.  Examples  of  such  benefits  are 
discussed  by  Atkinson  and  Paulson 
( 1 9?2), Atkinson  (1976),  and  Chant  and 
Atkinson  (1973).  The  improvements 
in  these  examples  cited,  come  mainly 
through  increased  precision  in  para¬ 
meter  estimation  for  the  student 
models.  One  benefit  comes  from  the 
increase  in  precision  in  the  optimi¬ 
zation  algorithms,  which  are  dependent 
upon  the  parameter  estimates.  A 
second  benefit  comes  from  qualitative 
changes  in  the  student  model  based 
on  the  additional  data.  As  an 
example,  Atkinson  and  Paulson  (1972) 
report  using  the  Random  Trials 
Increments  (RTI)  model  (Uorman,  1964) 
as  the  model  of  the  student.  As  is 
characteristic  of  the  RTI  model, 
variation  in  the  parameter  can  change 
its  qualitative  nature  from  that  of 
an  all-or-none  process  to  an  incre¬ 
mental  process  as  extremes.  By 
letting  the  parameters  be  estimated 
empirically,  the  most  representative 


mix  of  these  two  models  can  be  found. 
Further,  the  representation  improves 
with  added  data  from  additional  stu¬ 
dents.  This  principle  of  self¬ 
adjustment  through  empirically 
derived  estimates  controlling  a 
process,  could  be  generalized  to 
portions  of  the  system  other  than 
just  the  student  model. 

THE  BRIDGE  BET'.VESIJ  6.1  anu  6.2 
COI'TRACTIKG 

V/hat  we  have  just  described  are 
the  desireable  characteristics  of 
an  instructor  model  which  at  the 
present  time  does  not  exist.  First 
the  instructor  model  should  be  able 
to  deal  with  process  as  well  as 
structure.  This  means  that  its 
scheduling,  generative,  and  adaptive 
functions  should  reflect  dynamic 
estimates  of  the  students'  resource 
allocation  functions.  Second,  the 
instructor  model  would  also  need  a 
knowledge  representation  system 
capable  of  representing  process  as 
well  as  structure.  Third,  there 
would  be  a  need  for  adequately 
descriptive  student  models  and 
finally  a  set  of  algorithms  for 
self  modification. 

Currently  there  exists  several 
prototypes  of  the  characteristics 
we  have  just  described  in  the  6.1 
or  basic  research  literature.  Many 
of  the  efforts  were  funded  by  OER. 
Most  all  of  the  6.1  level  of  efforts 
tend  to  be  oriented  toward  certain 
technologies  such  as  the  representa¬ 
tion  of  knowledge,  speech  recognition 
etc.,  which  are  directed  toward  no 
single  particular  training  system, 
but  could  be  incorporated  into  sever¬ 
al.  Subsequent  levels  of  effort 
(6.2,  6.3,  etc.)  are  usually  more 
product  oriented  rather  than  technol¬ 
ogy  oriented.  That  is  to  say  that 
6.2,  etc.  contracts  are  usually 
organized  with  a  particular  end- 
product  in  mind,  e.g.  an  LSO  part- 
trainer.  Figure  7  illustrates  this 
situation. 

I’ote  that  there  is  a  gap  shown 
between  the  6.1  and  6.2  levels.  This 
is  to  illustrate  the  fact  that  most 
6.2  or  product-oriented  contractors 
do  not  vi ew  the  6.1  level  technolo¬ 
gies  as  being  in  a  "ready-to-implement 
state."  This  in  fact  would  be  the 
case  in  terms  of  the  technologies 
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required  for  the  instructor  model 
just  described.  Before  the  instruc¬ 
tor  model  could  be  incorporated  into 
an  LSO  trainer,  considerable  develop¬ 
mental  effort  would  be  required. 
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i'lgure 


Proposed  organiza¬ 
tional  structure 
of  R& D  efforts. 


Figure  7.  Present  organizational 
structure  of  R&D  effort 


Part  of  the  reason  for  the  gap  is 
that  for  the  most  part,  two  different 
groups  of  researchers  are  involved. 
The  6. 1  level  researchers  are  mainly 
university  scientists  with  theory  as 
their  goal,  while  the  6.2  levels  are 
industrial  scientists  with  time- 
frame  contingent  products  as  their 
goal.  Thus,  one  part  of  the  problem 
is  the  breakdown  in  the  crossfeed  of 
information  and  technological  needs 
caused  by  the  lack  of  having  both 
types  of  researcher  engaged  in  both 
levels  of  contracting. 

Figure  8  illustrates  one  possible 
solution  to  this  dilemma,  namely  a 
series  of  6.1  or  6.2  efforts  aimed 
at  bridging  the  gap.  These  efforts 
would  have  as  their  goals  the  syn¬ 
thesis  and  aggregation  of  the  more 
theoretical  technologies  into 
applied  technologies  such  as  the 
development  of  the  Instructor  model 
just  described.  These  intermediate 
levels  of  effort  would  have  to  cross, 
or  be  applicable  to,  a  number  of 
product-oriented  efforts  in  order 
to  be  cost-effective.  In  the  case 
of  the  instructor  model,  a  form 
general  enough  to  be  implemented 


in  several  trainers  would  need  to 
be  developed  so  that  the  cost  of 
development  could  bo  amortized  over 
several  trainers.  Finally,  those 
involved  in  those  efforts  should 
be  the  6.2  level  researchers  or 
at  least  personnel  that  would  be 
in  a  position  to  follow  and  consult 
in  the  creation  of  the  trainers 
throughout  their  various  stages 
of  development. 
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